Principal Software Engineering Manager - CoreAI

Share job

CoreAI is at the heart of Microsoft’s mission to revolutionize how software is built and experienced. We build the foundational platforms, services, and developer experiences that power the next generation of applications using Generative AI. Our work enables developers and enterprises to harness the full potential of AI to create intelligent, adaptive, and transformative software.

The Inferencing Control Plane team is responsible for building the distributed systems and orchestration frameworks that manage the lifecycle, routing, and execution of AI workloads at scale. We provide the critical infrastructure that ensures reliable, performant, and secure inferencing across heterogeneous environments and models.

This Principal Software Engineering Manager - CoreAI role is a unique opportunity to lead a team building the backbone of generative AI infrastructure at Microsoft!

Required/Minimum Qualifications

• Bachelor’s degree in Computer Science or related technical field AND 6+ years of technical engineering experience in languages such as C#, Java, C++, Go
  • OR equivalent experience.
• 4+ years of experience managing software engineering teams
• 4+ years of experience with distributed systems, microservices, and container orchestration technologies (e.g., Kubernetes, Docker)
• 4+ years of experience with CI/CD pipelines and developer tooling (e.g., GitHub, Azure DevOps)
• 4+ years of experience building and maintaining infrastructure platforms and product APIs with a focus on usability, extensibility, scale and performance

Other Requirements:

Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings:

• Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Preferred/Additional Qualifications

• Experience leading teams building infrastructure platforms, APIs, or orchestration frameworks
• Understanding of cloud-native architecture, asynchronous workflows, and service mesh technologies
• Familiarity with agent-based systems, workflow engines, or declarative programming models
• Ability to drive clarity in ambiguous, early-stage platform development
• Experience collaborating with product and design to shape abstractions and reference implementations
• Mentorship skills and a track record of fostering technical excellence and inclusive team culture
• Experience in AI/ML infrastructure, inferencing systems, or high-scale platform engineering
• Hands-on experience with Microsoft Azure services such as AKS, Azure Functions, and Event Grid, CosmosDb
• Demonstrated experience driving AI usage and adoption in platform or product development, including integrating AI capabilities into services or enabling AI-powered developer experiences

Software Engineering M5 - The typical base pay range for this role across the U.S. is USD $139,900 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay

Microsoft posts positions for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

#CoreAI

#azureopenai

#llm

#infrastructure

#distributedsystems

#inference

#technicalleadership

#AIPLATFORM

• Lead and grow a team of software engineers focused on building scalable, cloud-native inferencing services and orchestration layers.
• Partner with product, research, and platform teams to define requirements and deliver robust, extensible infrastructure platforms and product APIs.
• Drive architectural decisions and technical strategy for inferencing control systems, ensuring high availability, performance, and cost efficiency.
• Guide the team in designing and implementing distributed systems that support dynamic routing, load balancing, and execution of AI workloads.
• Ensure engineering excellence through code reviews, design reviews, and mentoring, fostering a culture of innovation and inclusion.
• Own the operational health of services, including on-call rotations, incident response, and continuous improvement of reliability metrics.
• Collaborate across CoreAI and Microsoft to align on platform integration, security, and compliance requirements.
• Champion developer experience and usability in SDKs and tooling, enabling seamless integration with agentic and generative AI platforms.