跳到正文
OpenBMB· @OpenBMB · X·· 8 小时前AI 评分45

开发者用 MiniCPM-V 4.7 1B 打造视障辅助 App PRTS

AI 导读

社区开发者 Rebel-Gloria 用 MiniCPM-V 4.7 1B 作为默认端侧视觉理解模型,做出了面向盲人和低视力用户的 iPhone/iPad 应用 PRTS。

正文 · 原文

📱 Point your camera. Hear what’s in front of you.

Community developer Rebel-Gloria built PRTS, an iPhone and iPad app for blind and low-vision users, with MiniCPM-V 4.7 1B powering its default on-device visual understanding. 💙

👀 Scene descriptions: Capture an image and hear a concise description of your surroundings.

📖 Read text: Photograph signs, notices, or printed pages and have them read aloud.

🎙️ Ask follow-up questions: Use voice input to ask about details in the current scene.

📳 Obstacle guidance: A separate LiDAR-based system provides spoken directions and rhythmic haptic cues on supported devices.

♿ Accessible controls: VoiceOver, Dynamic Type, physical buttons, Back Tap, and Siri Shortcuts offer multiple ways to interact. A dedicated low-vision mode retains the camera preview, scene descriptions, and text reading.

🔒 Routine scene descriptions and text reading can run offline with the built-in MiniCPM-V model. Optional online features and system speech recognition have separate connectivity requirements.

A thoughtful community exploration of visual intelligence for more accessible everyday experiences. 🛠️

📂 Explore PRTS:
github.com/Rebel-Gloria/PRTS
🚀 MiniCPM-V GitHub:
github.com/OpenBMB/MiniCPM-V
🤗 On-device model:
huggingface.co/openbmb/MiniCP…

#MiniCPM #OpenBMB #Accessibility #OnDeviceAI #OpenSource

来源:OpenBMB · x.com