Manual QA Tester (Blackbox Testing)

Job Title: Senior Speech & Audio ML Engineer Location: Remote Type: Full-time Salary Range: $2,500–5,500 USD / month Role Purpose We are looking for a Senior ML Engineer to build and ship core models for a speech-driven behavioral engine. You will own end-to-end modeling from raw, long-form audio and layered annotations to production inference. Responsibilities include: Designing audio features and embeddings. Training and evaluating a suite of models. Delivering reproducible pipelines that meet targets for accuracy, robustness, latency, and cost. Non-Negotiables Experience: 5+ years building production ML systems, including 2+ years in speech/audio. Speech & Signal Processing: VAD, diarization, segmentation, denoising, spectral features (log-mel/MFCC), prosody (pitch/energy), long-form audio handling. SOTA Audio Models & Embeddings: Wav2Vec2, HuBERT, wavLM (or similar); fine-tuning/self-supervised learning; contrastive/metric learning for downstream tasks. Data Engineering & Quality: SQL, Python data stack (Pandas/Polars), ETL for audio + metadata, stratified sampling, leakage prevention, feature stores. Evaluation Discipline: Golden sets, robust speaker/content splits, ROC/PR/calibration, fairness/bias checks, ablations, drift/shift detection on embeddings and audio quality. MLOps, Serving & Reproducibility: FastAPI/gRPC around HF/torchaudio models, experiment tracking (W&B/MLflow), artifact/model versioning, CI/CD, observability, scalable batch/streaming inference. Proven ability to create and document novel IP (methods, architectures, or training/eval techniques) with clear prior-art awareness. Nice to Have Tooling: SpeechBrain, Lightning, OpenSMILE/Praat, Kaldi/Conformer/Emformer, Label Studio. Multimodal Skills: ASR (e.g., Whisper) + paralinguistic features; emotion/prosody modeling; speaker embeddings (x-vectors, ECAPA-TDNN). Performance & Deployment: Quantization/distillation, Triton/CUDA basics, distributed training, real-time/streaming inference, on-device DSP (Rust/C++). Publications/Patents/Competitions: Demonstrating novel audio modeling work.

Job Title: LLM Engineer / Prompt & Agent Specialist Location: Remote Type: Full-time Role Purpose We are looking for a hands-on LLM Engineer to design, iterate, and evaluate prompts and agent behaviors. You will own system prompts, tool/function-call specs, schemas, guardrails, and evaluation processes to optimize accuracy, safety, latency, and cost for production LLM workflows. Key Responsibilities Build and optimize prompts for structured outputs and tool/function calling. Design and implement evaluation frameworks to measure performance, errors, hallucinations, latency, and cost. Develop safety mechanisms, guardrails, and adversarial/red-team testing for LLM workflows. Define clear human/machine handoffs with deterministic flows. Collaborate with engineering and product teams to improve LLM-based systems in production. Requirements 2+ years building with LLMs in production (or 4–6+ years engineering with LLM ownership). Expert in prompt engineering for structured outputs and tool/function calling. Fluency with LLM APIs (OpenAI, Anthropic, etc.). Strong evaluation skills: golden sets, rubric/auto-judge scoring, error taxonomies. Experience with safety, moderation, and rule-based constraints in LLMs. Nice to Have Experience with RAG, embeddings, and vector stores (Upstash, Pinecone). Fluency in Typescript. Familiarity with agent frameworks (Mastra, Langgraph, etc.). Experience with eval/prompt tooling and version management (OpenAI Evals, DeepEval). Knowledge of A/B testing, fine-tuning, and human-in-the-loop feedback loops. Observability and monitoring experience for LLM systems.

QA / Test Automation Engineer Location: Remote (LatAm time zones preferred) Type: Full-time Reports to: CTO About Mappa Mappa builds real-time voice-AI infrastructure that decodes over 100,000 behavior patterns per person—based on prosodic and linguistic signals—and turns them into actionable insights for hiring and beyond. Quality and reliability are core to our product—and we’re looking for someone who ensures both scale with us. Role Purpose Design and maintain test automation systems that guarantee the stability, performance, and trustworthiness of Mappa’s voice-AI platform. You'll lead the creation of robust test cases, frameworks, and regression suites integrated with our CI/CD pipelines. Key Outcomes (First 6 Months) Implement automated test coverage for all core application components (frontend and backend). Reduce manual regression testing time by ≥ 60% via automation. Detect and prevent critical bugs pre-deployment in at least 90% of release cycles. Integrate testing into our CI/CD workflows and ship reliable builds multiple times per week. Core Responsibilities Design and implement end-to-end, integration, and regression test cases for web and backend components. Build and maintain test automation frameworks using Python, TypeScript, Selenium, Puppeteer, and PyTest/Bun. Integrate testing pipelines with CI/CD tools (Jenkins, GitHub Actions) for automated validation. Collaborate with Engineering and Product to define test plans and acceptance criteria. Monitor test coverage, identify gaps, and proactively improve testing strategies. Triage bugs, write detailed bug reports, and contribute to root cause analysis post-deployment. Maintain test documentation and ensure testing processes are well-documented and repeatable. Must-Have Skills 3+ years of experience in QA engineering or test automation. Strong proficiency in Python and/or TypeScript for scripting and test development. Experience with test automation tools (e.g., Selenium, Puppeteer) and test frameworks (PyTest, Bun or similar). Solid understanding of CI/CD pipelines and automated testing in release workflows. Familiarity with test case management, regression testing, and performance validation. Excellent attention to detail and process ownership. Fluent in English. Nice-to-Have Experience in testing AI or voice-based applications. Familiarity with monitoring tools or observability stacks (e.g., Sentry, Datadog). Exposure to performance and load testing frameworks. Who You Are A quality guardian who builds systems to break things—before users ever see a bug. You bring structure to chaos, automate relentlessly, and think like both an engineer and a user. You’re the last line of defense and proud of it. Why Join Mappa Product-critical role: Your tests keep our behavioral AI system stable and trustworthy. Modern stack: Work with engineers shipping Python, TypeScript, CI/CD, and agents at scale. Remote-first: Join a distributed team across LatAm and the U.S.