ArgusSearch for Mobile – Artificial Intelligence for Keyword Spotting in all your photos!

Finding information in huge amounts of images over a variety of scenarios (e.g. documents, notes, white boards, screenshots, etc) has been a challenging task since many years. A solution is now available for your smartphone or tablet (Android) – ArgusSearch for MobilePlanet AI originally developed this new technology (pending patent) for Europe’s largest project to access our cultural heritage of ancient handwritten documents (approx. 11 Billion pages of historic handwritings), where our technology won several international awards for Keyword-Spotting.

Image 1: ArgusSearch on ancient documents

However, it turned out, that ArgusSearch can not only handle these fancy old scripts e.g. from Vatican library, it also works very robust on a large variety of other scenarios (e.g. modern script or machine printed).

Image 2: example for finding grandma’s recipe in my smartphone images

Image 3: examples for retrieving medical information from an image folder

With the idea in mind, that this could be a very helpful assistant for many smartphone users, our team has made the technology of ArgusSearch available for Android (iPhone is planned to come soon).

Technology – a glance behind the curtain

Its unique combination of latest state of the art Artificial Intelligence (AI) for text recognition and our new PerceptionMatrix (pending patent) technology – that are the ingredients making ArgusSearch so powerful. PlanetBrain (Planet’s AI system) first analyzes all text lines by creating an „internal meaning representation“ with a deep and recurrent Artificial Neural Network. While previous approaches create a searchable symbolic code (e.g. ASCII), we found a way to directly access the PerceptionMatrix (internal meaning representation) of PlanetBrain for search requests and content analytics.

Image 5: schematic view on our PlanetBrain analyzing the input text sequence (left image), avoid final
transcription but directly access our internal meaning representation (PerceptionMatrix) for any
requests (right image).

This approach of directly accessing the PerceptionMatrix of PlanetBrain instead of enforcing a transcription has significant advantages for handwritings, difficult machine printed texts and speech. Or in short, every time when a transcription could be ambivalent, the PerceptionMatrix is the best solution.

Image 6: the spoken phrases „wrecking nice beaches“ and „recognize speeches“ sound quickly
spoken very similar while having a different spelling and of course a different meaning (left image). Our
PerceptionMatrix preserves both interpretations (right image). Both search requests would be found in
a spoken sequence of course with different probabilities.

Additionally our technology supports Regular Expressions to a good extend. That makes it possible to search for certain pattern, e.g. dates or phone numbers additionally to the search for words and phrases.

Image 7: example for a search by means of Regular Expressions, all results found are listed according
to matching quality, multiple results per page are combined.

Previous to any search requests an indexing run over all images creates an index database for all these images. Indexing describes the process of analyzing all images for texts and reading these texts by our AI, PlanetBrain. This indexing is performed in background processing solely on device (no data are exchanged with any external system e.g. cloud). The indexing may take a while depending on the performance of your device and the amount of images to be indexed. The progress will be visible in the graphical user interface. Recently we support all languages based on latin characters, we recommend to formulate your requests using standard characters only (e.g. use „A“ instead of „Å“). Please get your own impressions about that sophisticated piece of technology here and let us know your feedback!
For more information please check out our website: or ask for details.

Download PDF