App Security Expert

About the work

Build realistic application security fixtures and deterministic tests for AI evaluation. This remote contract focuses on LLM risks, security architecture, and reliable assessment of vulnerable versus secure code; no prior AI experience is required.

Responsibilities

• Design and implement small, purpose-built application fixtures with embedded security vulnerabilities or secure solutions, specifically addressing OWASP LLM risks such as Sensitive Information Disclosure and Excessive Agency.
• Develop deterministic test cases and security-context manifests to robustly assess application security posture.
• Create detailed hidden answer keys to enable accurate AI or scanner evaluation of vulnerable versus secure code.
• Evaluate and iterate on pipeline methodologies to ensure synthetic tasks and fixtures are challenging yet realistic for AppSec assessment.
• Document technical insights and methodologies, ensuring clarity in written feedback and communication.
• Collaborate through clear and thorough written and verbal communication to share findings and review peer contributions.
• Engage with agent-based security architecture and contribute to continuous improvement of the evaluation pipeline.

Preferred background

• Extensive experience (ideally 5+ years) in application security, secure software development, or CyberSWE roles.
• Strong understanding of OSWE, LLM security, AI-powered static analysis (AI-SAST), and AppSec best practices.
• Proven track record designing or evaluating security architecture for traditional and agent-based applications.
• Expertise in building, securing, and testing application codebases, including identifying and remediating advanced vulnerabilities.
• Exceptional written and verbal communication skills, with a focus on precise technical documentation and collaborative feedback.
• Experience with synthetic security fixtures and deterministic testing approaches is highly desirable.
• Demonstrated ability to deliver thoughtful insights and actionable security recommendations in a remote contribution context.