Software Engineer - Pre-training Systems

Magic’s mission is to build safe AGI that accelerates humanity’s progress on the world’s most important problems. We believe the most promising path to safe AGI lies in automating research and code generation to improve models and solve alignment more reliably than humans can alone. Our approach combines frontier-scale pre-training, domain-specific RL, ultra-long context, and inference-time compute to achieve this goal.

About The Role

As a Software Engineer on the Pre-training Systems team, you will design and operate the distributed infrastructure that trains Magic’s long-context models at scale.

This role focuses on large-scale model training across massive GPU clusters. You will work at the boundary between deep learning and distributed systems, ensuring that training runs are performant, reliable, and reproducible under extreme scale.

Magic’s long-context models create non-trivial systems challenges: sustained memory pressure, communication overhead across thousands of devices, long-running jobs that must survive failures, and efficient sequence packing under hardware constraints. You will own the systems that make large-scale pre-training stable and fast.

What you’ll work on

• Scale distributed training across large GPU clusters (data, tensor, pipeline parallelism)
• Optimize communication patterns and gradient synchronization
• Improve checkpointing, fault tolerance, and job recovery systems
• Profile and eliminate performance bottlenecks across compute, networking, and storage
• Improve experiment reproducibility and orchestration workflows

Looking to advance your Development & Programming career with relocation support? Explore Development & Programming Jobs with Relocation Packages that include comprehensive packages to help you move and settle in your new role.

• Increase hardware utilization and training throughput
• Collaborate with Kernels and Research to align model architecture with systems realities
  
  

What we’re looking for

• Strong software engineering and distributed systems fundamentals
• Experience training large models in multi-node GPU environments
• Deep understanding of parallelism strategies and performance trade-offs
• Experience debugging cross-layer issues in production ML systems
• Strong ownership mindset and ability to operate critical infrastructure

Discover our full range of relocation jobs with comprehensive support packages to help you relocate and settle in your new location.

• Track record of improving performance or reliability of large-scale systems
  
  

Compensation, benefits, and perks (US):

• Annual salary range: $225K - $550K
• Equity is a significant part of total compensation, in addition to salary
• 401(k) plan with 6% salary matching
• Generous health, dental and vision insurance for you and your dependents
• Unlimited paid time off
• Visa sponsorship and relocation stipend to bring you to SF, if possible

Interested in relocating to United State? Check out our comprehensive Relocation Jobs in United State page with detailed relocation packages and benefits.

• A small, fast-paced, highly focused team
  
  

Magic strives to be the place where high-potential individuals can do their best work. We value quick learning and grit just as much as skill and experience.

Our culture

• Integrity. Words and actions should be aligned
• Hands-on. At Magic, everyone is building
• Teamwork. We move as one team, not N individuals
• Focus. Safely deploy AGI. Everything else is noise
• Quality. Magic should feel like magic
  
  

Compensation Range: $225K - $550K