Features

Transcript audio notes - not in 2026

13d agoPending

Consider doing IF the Record audio/sound to a note is done. This is dependant on that. Original request was to record audio, transcript came as a related idea.

4 comments

  1. PikaNotes marked this as Pending6d ago
  2. PikaNotes6d ago

    Transcription of audio clips was tested but not included in the 1.1.0 release for now as they turned out to be unreliable.

    We tested whether PikaNotes could turn audio clips into text using the speech recognition built into Android. The goal was transcription that works on the phone itself, so that recordings never leave the device. The results were not good enough to ship, so transcription will not be part of the upcoming release.

    What we found:

    - Even short, clearly spoken English sentences often went unrecognized. On-device recognition does not support Finnish on our test phone. - Network recognition handles short clips only. Google's network-based recognition handled short Finnish phrases well, but longer recordings failed, because the interface is designed for brief voice commands rather than full recordings. - Language must be chosen manually. Recognition forces speech into the selected language, so Finnish spoken with English selected for transcribing produced unrelated English text. - Our side works. PikaNotes correctly passed the recording to the recognizer, and the recognizer read the file. The limitations lie in the recognition services, not in how PikaNotes delivers the audio.

    Shipping transcription today would mean either poor results or sending audio to Google, and we don't want either as a default. Offline models with good Finnish quality exist, but they are large and slow on a phone, which conflicts with keeping PikaNotes light and fast.

    In the future, maybe:

    We may revisit the feature later in one of two ways. One is optional network transcription that splits recordings into short segments and clearly asks for consent before any audio is sent. The other is on-device transcription, if Android adds Finnish support. In the meantime, the keyboard's built-in voice typing can be used to dictate directly into a note.

  3. PikaNotes marked this as Planned7d ago
  4. PikaNotes marked this as In Progress7d ago
  5. PikaNotes moved this from Bugs to Features12d ago
  6. PikaNotes12d ago

    This is related to audio notes and this is dependant on that. (https://pikanotes.userjot.com/board/p/record-audio-sound-to-…)

    This can be done as a separate feature from the audio notes or with it, but not separately.

    Some preliminary specs below.

    Transcription of voice recordings

    Use cases: - Turn a dictated idea into editable text under the recording. - Skim a long recording as text instead of listening to it.

    - Transcribe feature to an audio element. The result is inserted as normal text paragraphs under the audio element (editable, searchable and backed up).

    Other considered alternative ways - Phone's keyboard microphone (eg. Gboard voice typing) already dictates text to the app without any new permissions, but it does not transcribe saved recordings.

    Options

    A) Cloud API (OpenAI, Google Cloud) + Makes very good transcriptions on many languages - Requires internet connection, possible privacy issues. - Has running costs ~0.003–0.006 USD/min paid by the developer by default OR user enter own API key but gets slightly technical for the end user - Several issues against current way how PikaNotes Works: probably needs own server as API key cannot be embedded in the app, user authentication and usage limits, developer becomes data controller for audio (GDPR) etc. => Not practical.

    B) On-device model (.e.g Whisper) + Fully offline, no internet requirement, no running costs - Probably requires considerable amount of space from phone. E.g. Finnish quality models depend on size (~60 MB weak or ~200-500 MB for good ones). - Resource heavier than API: slower on older phones - Model delivered as an optional Play asset pack, downloaded by Play Store when the user enables transcription (to be verified: app itself makes no network connection). - Native code, must meet Play's 16 KB page size requirement. => Possibly practical

    C) Android's built-in on-device speech recognition + No size increase, no running costs - Designed for short phrases, not long files - File inputs require Android 13+, Finnish support varies by phone - Must use on-device recognizer only, because the default one may send audio to Google (app has no internet, but no-internet promise must be "in spirit").

    D) Share recording to another app that user wants to use for transcription + "Cheap" way around, requires only sharing the audio - Low value to the use case: text must be copied back manually

    Internet requirement - Cloud API: Internet requirement required - On-device model: Internet not required if asset pack delivery is confirmed to need no app permission - Android's built-in on-device speech recognition: Internet not required if limited to on-device recognition - Sharing the recording to another transcription app: Internet not required.

    Privacy policy - Cloud API: Major changes as audio must be sent to third party, data controller role etc. - On-device model: No major change beyond the voice recording feature. - Android's built-in on-device speech recognition: No major change beyond the voice recording feature. - Sharing the recording to another transcription app: No major change or possibly none.

    Security - Cloud API: New attack surface (server, API key, accounts). Audio leaves the device. - Other options: No major change. Audio and text stay on the device and in encrypted backups, but the sharing option depends on what app is used for it.

    App size increase - On-device model increases approx 1-2MB for native library and 40-500 MB for on demand downloads. - No big change on other options.

    1. PikaNotes12d ago

      Narrowing down the options.

      If this is done, the principle should be to use on-device only and never cloud because privacy and keeping data to at user's device is integral part of PikaNotes.

      For that reason option A (Cloud API) is ruled out. It brings costs, complexity, possibly more legal liabilities and worsens the privacy.

      Option D (Sharing recordings to other app) is something user could do anyways as long as recordings are downloadable. It gets the job done, but its cumbersome to user and not really much of a "feature" if user does all the work.

      So, if this feature is done, it would be probably a selection between on-device model (B) or Andrdoid's build-in on-device model (C).

    2. PikaNotes12d ago

      Comparison table of the realistic options based on a quick research. Neither look very appealing. Android's own dictation may be better for quick testing and to see whether this should be improved, but it doesn't seem to be very reliable especially in foreign languages and I've noticed it in Gboard also. Still, downloading a language pack of some 300-1000 MB just for transcripts is impractical as it would literally multiply the size needed possibly over ten times just because this one feature.

      So, possibly a test with Androids own dictation is implemented at some point but it may not be very reliable.

      Another possibility would be to allow API usage (ruled out earlier) for advanced users to whom this might be an important feature as it provides best results in terms of dictation accuracy, but since it requires internet and affects privacy, its not something to do at least of now.

      • img/i_17r2mxmzzr3hzyylg5k58a5q