Copy-Paste Detector for 150+ Languages on JavaScript

on YouTube ↗29m

" summary

A JavaScript tool detects copy‑paste across files by tokenizing code (using CodeMirror for 150+ languages), filtering comments/noise, and hashing sliding windows of tokens with Rabin‑Karp substring hashing to find matches. It reports duplicate file/line snippets, integrates with CI systems (including Jenkins and XML/HTML reports), attributes lines via git blame, and is distributed as an npm package with ~200k downloads last year.

AI-written summary · about

" chapters (12)

1. 0:00Introduction
2. 3:38Copy-paste tools problems
3. 6:08Robin-Karp for duplicates
4. 8:55App architecture for search
5. 11:43Token service for languages
6. 13:51Using syntax parsers
7. 16:25CodeMirror for 150+ languages
8. 18:14Development status
9. 20:26Usage stats and community
10. 21:30Report features
11. 23:00Future improvements
12. 27:10Ignoring code blocks

" related talks

• 25mAuto-Detection of Copied Code from StackOverflowAndrew Walker · DevConf.CZ 2020
• 40mHow To (Semi-)Automate JavaScript RefactoringGiles Bowkett · Fluent 2013
• 43mJavaScript ❤ binary dataKirill Cherkashin · HolyJS 2018 Piter
• 37mMachine Learning on Source CodeFrancesc Campoy Flores · Scale By The Bay 2018
• 32mCan You Read Me? Creative JavaScript to Make Computer Science FunMariko Kosaka, Suz Hinton · JSConf EU 2017
• 50mJavascript code formattingChristopher Chedeau · React London 2017
• 40mRefactoring large JavaScript codebasesMichele Riva · jsday 2022
• 26mJavaScript Tools at Scale Using Type InformationMark Knichel · JSConf EU 2014