Backend Engineer _ AI Gateway

Position Information

• Job Title: Backend Engineer _ AI Gateway
• Contract Period: 1 yr.
• Work Hours: 9-6 local time
• Work Location:

-700 Sylvan Ave Englewood Cliffs, NJ 07632 (through September 2026)

-6625 Excellence Way, Plano, TX 75023 (effective October 2026 onward)

• Posting #:
• Person in needed: 1

JD Details

We are building an AI Gateway — a high-performance, secure intermediary layer that routes, inspects, and governs all traffic between enterprise clients and LLM/AI service providers. This role requires deep expertise in network programming, async Python, and cloud infrastructure to design and operate a latency-sensitive, policy-driven gateway that handles massive concurrent streaming connections at scale.

What we are looking for

• Async Python & Network Programming (3+ yrs) – Strong hands on experience with asyncio, aiohttp/anyio, low level TCP/UDP sockets, WebSocket & SSE servers, TLS handshake, connection pooling, keep alive, and zero copy buffering.
• Streaming & Large Payload Handling – Ability to manage chunked transfer, bidirectional streaming, back pressure and high throughput data paths for real time LLM inference.
• Protocol Knowledge – Working familiarity with HTTP/2 & HTTP/3 (multiplexing, HPACK), gRPC, and occasional custom protocol parsers.
• FastAPI Based Backend (3+ yrs) – Design, develop and test production grade REST/Streaming APIs; pragmatic use of Pydantic, dependency injection and OpenAPI docs.
• Containerization & Kubernetes – Docker, AKS/EKS/GKE deployment experience; HPA/VPA scaling, rolling updates, and service mesh (e.g., Istio/Envoy) basics.
• Async Messaging & Event Driven Design – Production use of Redis Streams, RabbitMQ or Kafka for decoupled task queues, retries and back off logic.
• PostgreSQL & Vault – Advanced query tuning, partitioning, PgBouncer pooling, plus HashiCorp Vault (dynamic secrets, transit encryption) for key management.
• Cloud Networking & Security – L4/L7 load balancing, TLS termination, reverse proxy (NGINX/Envoy), mTLS between services, and automated cert lifecycle (ACME/Let's Encrypt).
• IaC & CI/CD – Terraform modules for multi env infra, plus GitHub Actions/GitLab CI pipelines with container image scanning and canary/blue green deployments.

Work experience desired

• Enterprise Security Gateways – Integration with CASB/DLP/WSS or similar edge security platforms; centralized logging & audit trails.
• IAM & SSO – Configuring Azure AD/Entra ID, Okta, SAML 2.0 or OIDC SSO flows; managing Conditional Access & SCIM provisioning.
• AI/LLM Pipelines – Building prompt engineering workflows, PII anonymisation, and monitoring Azure OpenAI (or comparable) usage, rate limits & cost.
• API Gateway Operations – Custom webhook/REST integrations, circuit breaker patterns, rate limiting, auto retry and health checking.
• Observability & FinOps – End to end monitoring with Datadog, Prometheus/Grafana or Azure Monitor; proactive cost optimisation and 24/7 incident response (SLA/SLO).

New JO -

Position Information

• Job Title: Backend Engineer _ AI Gateway
• Contract Period: 1 yr.
• Bill rate: ~ $14,500.00/mo.
• Work Hours: 9-6 local time
• Work Location: 6625 Excellence Way, Plano, TX 75023
• Posting #: 4000096343
• Person in needed: 1

Bilingual Korean Required

JD Details

About the Role

We are building an AI Gateway—a high‑performance intermediary that routes, inspects, and governs traffic between enterprise clients and LLM/AI service providers. In this role you will own the entire full‑stack lifecycle of the gateway, from core Python API service design and implementation to containerization, deployment, monitoring, and day‑to‑day operations.

While you can consult network and security specialists for architecture reviews, threat modeling, and policy definition, the application code, testing, CI/CD pipeline, and production operations will be developed and maintained solely by you. You’ll have full technical autonomy to design, build, and scale a robust, production‑grade system.

What we are looking for

• Experience: Around 5 years of professional backend development experience, with a solid track record of building and operating production-grade services.
• Async Python & Core Web Frameworks (3+ yrs): Deep understanding of asyncio and hands-on production experience with FastAPI (Pydantic, dependency injection, OpenAPI).
• Network Protocol & Streaming Fundamentals:
• Strong grasp of HTTP/1.1, HTTP/2, and SSE (Server-Sent Events) / Chunked transfer encoding essential for real-time LLM inference streaming.
• Practical understanding of the TCP/IP lifecycle, connection pooling, keep-alive, and managing back-pressure.
• Database & Async Messaging:
• Proficiency in PostgreSQL (query tuning, connection pooling via PgBouncer).
• Experience with message queues or event streams (e.g., Redis Streams, RabbitMQ, or Kafka) for decoupled task processing.
• Containerization & CI/CD:
• Docker, Kubernetes (basic workload management, deployment, HPA), and setting up CI/CD pipelines (GitHub Actions / GitLab CI).

Work experience desired

• Collaborative Security & Infra Awareness: Willingness to work with internal security/network teams on NGINX/Envoy reverse-proxy setup, mTLS, or IAM/SSO (OIDC/SAML) integrations.
• AI/LLM Integration: Experience handling LLM API payloads, managing rate limits, or implementing simple proxy/gateway patterns for AI services.
• Infrastructure as Code: Basic familiarity with Terraform for multi-env infrastructure management.