VScan
A visual perception layer for the blind
New in version 0.2.3
- There is now a standalone editor for entering the system prompt and user prompt. This editor has a large text field, which should make it easy to work with long and complex prompts.
- Various UI improvements and bug fixes.
- Various UI improvements and bug fixes.
This is a little project of mine aiming to research how vision LLMs could help out blind people on travel and in their every-day life by substituting eyesight for various visual tasks. VScan turns your smartphone's camera into a device for visual perception. You can define various optical cognitive functions, like looking for objects, signs, evaluating a scene or simply mediating visual impressions. You can afterwards use these functions on the camera view, just like a sighted person would use their eyes to achieve a specific goal in the physical world.
Each cognitive tool consists of two major parts:
- The camera to be used - front / back, as well as camera parameters - resolution, flashlight etc.
- The prompts used for LLM processing. LLM is the bridge between raw pixel data and your interpretation of it, and in the user/system prompt, you can specify what exactly are you interested in for the particular function and how should it be communicated, as well as the LLM model that should be used.
Camera input in combination with an LLM processing prompt forms a cognitive function, which can be used to serve various visual tasks.
VScan is open-source software. Visit the project's official repository to learn more about its background, motivation, specific usage examples and setup instructions.
- Author: Rastislav Kish
- License: GNU General Public License v3.0 only
- Issue Tracker
- Source Code
- Build Metadata
- Reproducibility Status
Versions
Although APK downloads are available below to give you the choice, you should be aware that by installing that way you will not receive update notifications and it's a less secure way to download. We recommend that you install the F-Droid client and use that.
Download F-Droid-
arm64-v8aarmeabi-v7ax86x86_64This version requires Android 7.0 or newer.
It is built and signed by the original developer, and guaranteed to correspond to this source tarball.
Download APK 6.6 MiB PGP Signature | Build Log



