CAMBRIFY

Objective

To develop a prototype for "Cambrify", a specialized AI educational assistant, focusing on creating a natural, voice-driven conversational experience for learning complex subjects like physics.

Cambrify AI Assistant

Cambrify is a proof-of-concept SaaS application featuring an AI assistant powered by a fine-tuned OpenAI model (ft:gpt-4o-mini-2024-07-18:cambrify::BnpNonuB). It leverages Microsoft Azure's Speech Services to enable seamless, real-time voice conversations, making learning more interactive and accessible.

What we've done

• Fine-Tuned Model Integration: Configured the application to use a custom-trained OpenAI model for specialized, context-aware responses.
• Real-Time Voice Interaction: Implemented robust, two-way voice communication using Azure's text-to-speech (en-US-JennyNeural) and speech-to-text services.
• Live Conversation Mode: Developed an immersive chat mode that features a live transcript and hands-free operation.
• Natural Interruption Handling: Engineered the core logic to allow the user to interrupt the AI's speech by talking or typing, immediately pausing the AI to create a fluid, human-like dialogue.
• Development Environment Setup: Resolved local development hurdles by enabling HTTPS via an ngrok tunnel, satisfying browser security requirements for microphone access.
• Component-Based UI: Built the user interface with React and Vite, including responsive components for the chat and live-mode displays.

Creative Process

• Research: Connected the app to the fine-tuned model and Azure Speech SDK.
• Sketching: Built the basic chat interface and integrated text/voice inputs.
• Digital Design: Implemented Live Chat Mode and speech management system.
• Refinement: Debugged, resolved HTTPS/SSL issues, fixed module errors, expanded transcript area.
• Finalization: Stable, feature-rich prototype handling interruptions gracefully.