Senior VR/XR & Computer Vision Software Engineer

Job Posting: Senior VR/XR & Computer Vision Software Engineer With Experience Building Full-Stack AI-Driven Immersive Systems Location: Remote (AWS-Cloud Focused) Infrastructure: Ubuntu 24.04, AWS (G5/C6g), Red5 Pro (Streaming Server, TrueTime, and Terraform), NGINX Stack: TensorFlow, Three.js / Babylon.js, React / Vue.js / Django, Node.js What You’ll Build & Prototype AI‑Driven Computer Vision & Facial Tracking

• Rapidly prototype and refine facial landmark detection pipelines using MediaPipe, ARKit/ARCore, or custom TensorFlow/PyTorch models.
• Build ML‑based blendshape mapping systems (regression, PCA, neural networks) based on live-streaming data that drive expressive 2D and 3D full-body representation of a person.
• Experiment with new Cross-Validation/Machine Learning techniques and integrate them into real‑time pipelines with sub‑100‑ms latency.
• Use AI tools to accelerate model iteration, data labeling, and performance tuning.
• AI Integration, implementation, and development of fine-tune tracking models using Google MediaPipe, TensorFlow, ARKit, and NVIDIA Maxine.

3D Graphics, Avatar Animation & Blender Integration

• Build expressive 3D environments and animation systems using Three.js.
• Work directly with Blender to create, modify, and export glTF avatars with morph targets and PBR materials.
• Prototype new animation techniques, shader effects, and environment interactions quickly.
• Shader Programming by writing custom GLSL/HLSL shaders to handle realistic skin textures, PBR materials, and real-time lighting within the virtual space.
• 3D Environment Development: Architect and render realistic 3D worlds using Three.js, Babylon.js, and Blender ensuring high-performance execution via WebGL and WebGPU.
• Collaborate with artists and designers in a fast feedback loop.

Real‑Time WebRTC & Distributed Media Systems

• Architect WebRTC pipelines for video, audio, and data channels.
• Integrate with our AWS‑hosted Red5 Pro cluster, including Streaming Manager, TrueTime Meeting, Terraform, and autoscaling components.
• Prototype new networking strategies, optimize ICE/TURN/STUN flows, and experiment with adaptive bitrate logic.
• Use AI‑assisted debugging and monitoring tools to accelerate iteration.
• Develop live real-time avatars and optimize the pipeline for mapping 2D webcam/camera feeds to 3D meshes using Facial Action Coding System (FACS) and morph target influences.

Startup‑Style Collaboration

• Work closely with founders and cross‑functional teammates to shape product direction.
• Prototype features rapidly, test them with real users, and iterate based on feedback.
• Contribute to architectural decisions, technical strategy, and long‑term platform vision.
• Thrive in a culture that values creativity, autonomy, and speed.

Role Overview We are seeking a high-caliber Senior Full-Stack Software Engineer/Developer with a specialization in VR/XR and Computer Vision to lead the engineering of our "metaverse-class" social immersion platform. This role is at the bleeding edge of technology: you will build the systems that overlay live-streamed facial video onto computer-generated and live-captured 2D and 3D images of a person, creating a seamless "hybrid" user representation in realistic 3D worlds. As a Full-Stack lead, you will own the entire vertical—from the Ubuntu/AWS backend and Red5 Pro streaming pipelines to the TensorFlow tracking models and the React/Three.js frontend environments. 1. Full-Stack Responsibilities & Frameworks · Frontend (Web & VR/XR): Develop high-performance, immersive interfaces using React or Vue.js. Implement the 3D world layer using Three.js or Babylon.js, ensuring the hybrid-user representation renders with sub-100ms latency via WebGL. · Backend & API: Architect robust server-side logic using Django or Node.js to manage user sessions, world state synchronization, and secure data persistence, Red5 Pro streaming server, NGINX (reverse proxy, SSL, caching), Docker/Kubernetes (scaling), and Redis/PostgreSQL for session/state management. · Streaming Infrastructure: Manage the Red5 Pro TrueTime Meeting and Stream Manager deployment on Ubuntu 24.04, handling the ingestion and distribution of ultra-low latency WebRTC streams. · Native Integration: Maintain and extend client functionality for Windows and Mobile using C# or C++. 2. AI, Computer Vision & Prototyping · CV/ML Pipeline: Develop and integrate TensorFlow (and TensorFlow.js) models for real-time facial landmarking. Use Cross-Validation to ensure tracking accuracy across all user demographics. · AI-Expedited Dev: Utilize AI tools (GitHub Copilot, Gemini, Cursor, v0.dev, etc.) to accelerate full-stack feature delivery and Luma AI for rapid 3D environment asset generation. · Hybrid Rendering: Write custom GLSL/HLSL shaders to blend live video textures onto 2D and 3D meshes realistically. 3. Methodology & Security (Secure SDLC) · Secure SDLC: Apply a "Security by Design" philosophy across the full stack or critical components of the stack. Perform threat modeling on WebRTC signaling, implement secure OAuth2/OIDC flows, and manage NGINX for SSL termination and WAF protection. · Agile Methodology: Lead and participate in Agile workflows (Scrum/Kanban), translating complex user stories into technical requirements and iterative releases. · DevOps: Orchestrate AWS infrastructure (EC2, S3, CloudFormation) and optimize the Ubuntu server environment for high-throughput media processing. 4. Technical Toolbox Category

Requirements

Frontend Frameworks React, Vue.js, Angular, Three.js, Babylon.js Backend Frameworks Django, Node.js, FastAPI Streaming & Network Red5 Pro (TrueTime & Streaming Manager API), WebRTC, RTSP, NGINX, Ubuntu 24.04 AI & CV TensorFlow, OpenCV, MediaPipe, Scikit-learn Cloud (AWS) EC2 (G5/C6g), S3, Route53, AWS Wavelength (5G Edge) Languages Python, JavaScript/TypeScript, Java, C#, C++ Methodologies Agile, Secure SSDLC DevOps AWS CloudFormation/Terraform 3D/Graphics Three.js, GLSL Shaders, UV Mapping, Blender Databases Mongo, MySQL, NoSQL, Redis/PostgreSQL Key Qualifications · Experience: 7+ years in Full-Stack development, with at least 3 years focused on Real-Time Communications (RTC), VR/XR, WebRTC, and Computer Vision. · Architecture: Proven ability to design and deploy scalable, secure client-server cloud-native applications on AWS, VPS, and bare-metal servers. · Problem Solving: Deep understanding of the synchronization challenges between live video streams and 3D coordinate data. · Security: Solid experience implementing security best practices throughout the software development life cycle. · Portfolio demonstrating AI-driven video manipulation or Web3D immersive environments.

How to Apply

Submit your resume to [email protected] with a link to your GitHub or portfolio. We are specifically looking for candidates who can demonstrate a completed project involving real-time video integration using Red5 Pro implementations on AWS platform, TensorFlow-based tracking, 3D web rendering, and a secure backend architecture. Pay: $30.00 - $40.00 per hour Work Location: Remote Apply tot his job Apply To this Job