Deep learning module recognizes video sounds
summary
Project pipelines automated transcripts and background-sound labels using Azure Video Indexer for timestamps, Azure Databricks for custom ML and orchestration, spectrogram images fed to CNNs and Hugging Face NER to inject fresh named entities from news feeds. Goal aims for WCAG-level reliability (>99.9%) while delivered results report overall word accuracy ~97%, punctuation ~94%, numeric accuracy ~92% and special-word (names/places) ~81%; background-sound classifier uses 4s chunks and CNNs on spectrograms and is integrated into final caption outputs.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 3:21Video transcription types
- 5:56Tech stack and workflow
- 7:47Sound detection model
- 11:01Sound cluster analysis
- 12:29Spectrograms with neural nets
- 14:18Improving indexer accuracy
- 15:45Keyword extraction approaches
- 17:57Hugging Face for NER
- 19:23Model accuracy and next steps
- 21:12Q&A
related talks
25mLooking at Sound
28mHow to use Deep Learning to process audio data and classify sounds.
01h 00m5 Levels of AI in Microsoft Azure
45mAuditory Intelligence : beyond speech recognition
22mMachine learning in Education
50mCognitive Services Extravaganza!
36mVideo analytics at scale: Deep Learning, Computer Vision, ML & Spark
46mBuilding computer vision applications Q&A