The learning curve for video editing has never been shorter and shallower. As a result, the very profession of Video Editor may go extinct in the future. Video editing is being taken over by film directors, producers, and script editors.
Video editing has boiled down to simple “copy”, “cut” and “save”, as editing technology and digital media continue to advance, simplifying video editing. Complex video editing requiring video switching and coding skills, has ceased to exist. Flash drives have replaced analog Betacams, while codecs are now using a limited number of standards. One of the most commonly used standards is H.264.
Video production is now nearing a new turning point due to the development of neural networks.
Now finding video snippets is easy. About one third of the work of a video editor is searching for video sequences in the source file. They have to watch it through multiple times, looking for objects in the context of the voice-over narration, which is quite boring. It got me thinking why not develop software that could go over a folder with source files, recognize objects, and add them into the database? In order to search for video snippets the search word would be entered, such as “Sun” for example, and everything related to this word would somehow be moved to the video editing program.
No sooner said than done. After a while I submitted my app called Videoindex to the Mac App Store.
Videoindex offers the user to specify a path to the source folder, choose one of the two integrated pre-trained machine learning models Core ML (or use their own models, if any), and start scanning their video libraries.
It recognizes objects in video files and saves information about the path to a specific file and the timecode of the object found. Once the scanning is complete, the user can use the search window to retrieve all snippets found that correspond to the relevant search request.
The snippets can be exported to XML format that is used specifically to transfer file sequences and that is supported by virtually all editing software. As a result, everything you’ve been looking for, will be displayed in sequences in your project in just a couple of clicks.
What took you half a day before, can now be done within minutes! Similar (or the same) recognition technology is used in iOS for object recognition within photos and for improved navigation throughout albums in your iPhone.
Video objects are recognized with the use of machine learning frameworks. The app comes with two models: YOLOv3 and Resnet50. You can use them together or separately, or use your own Core ML model files. To manage the models, open the settings window with three slots for model uploading. The first slot is to upload YOLOv3, the second one is to upload Resnet50, and the third one is for your own Core ML model files.
Videoindex tracks the status of the catalogues added for scanning, i. e. if you delete or add any files to one of the folders, the Videoindex database will be updated next time you run it. If there are any new files in a folder that has already been scanned, its status will change to “Not scanned yet” to enable you to scan the new video files.
Note that the search and indexing efficiency depends on that of the pre-trained models that contain information about the object to be recognized. As the number of model files increases daily, there appear communities offering users to download model files in a wide range of formats, such as Kaggle, or just a set of images Dataset that are used to create these pre-trained models.