whiteboard unique keyframes

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
bvsp
Posts: 5
Joined: 2020-01-31T09:06:05-07:00
Authentication code: 1152

whiteboard unique keyframes

Post by bvsp »

I want to identify unique keyframes from a video of a lecture (whiteboard).
Let's say I extract frames every 1 second and there are a total of 1000 frames.

Frames will be additive, meaning that:
- frame 1 will have just one word written in the whiteboard
- frame 4 will have two words
- frame 10 will have five words
- frame 100 will be the point of the most writing on the canvas, just before the erasing
- frame 105 will have one word
- frame 110 will have two words
- frame 250 will be the point that contains the most writing on the canvas, just before the erasing
etc.

The output should be frame 100 and frame 250 (keyframe 1 and 2 respectively).

How can the spatial information be considered?
I'm thinking of calculating the frame difference with compare ( http://www.imagemagick.org/Usage/compare/ ), but comparison should be made between frame 1-100 and 100-250 for example and not frame 5 with 105.

snibgo
Posts: 12897
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: whiteboard unique keyframes

Post by snibgo »

What version of IM, on what platform?

You want the frames with the most writing, just before erasure. If your board really is white, then these are the darkest frames. As more words are written, the frames become darker. Then it suddenly becomes light again.

So your script takes the mean of each frame. This mean will decrease. When it suddenly increases, the previous frame is a keyframe.
snibgo's IM pages: im.snibgo.com

bvsp
Posts: 5
Joined: 2020-01-31T09:06:05-07:00
Authentication code: 1152

Re: whiteboard unique keyframes

Post by bvsp »

Linux, version 6.9.7

What about if there is occlusion by the writer? This would confuse the above method.
Also how do you suggest to get the darkest frame? histogram?

snibgo
Posts: 12897
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: whiteboard unique keyframes

Post by snibgo »

To get the average darkness of a frame:

Code: Select all

convert frame.png -colorspace Gray -format %[fx:mean] info:
This gives a number between 0.0 (black) and 1.0 (maximum white).

If the writer occludes the whiteboard, there will be a major darkening of the frame, much more than writing a word, so readily detectable. But is the erasure done by a person standing there? Then you have problems.

If you post some sample frames (upload to anywhere on the web, and paste URLs here), that would help.
snibgo's IM pages: im.snibgo.com

bvsp
Posts: 5
Joined: 2020-01-31T09:06:05-07:00
Authentication code: 1152

Re: whiteboard unique keyframes

Post by bvsp »

Thanks snibgo!
Hm maybe I can check the gradual darkening of the picture and disregard the frame (the one with the person occlusion) if it happens to be not within some threshold.
The erasure will be done by the person standing there I guess.
I will try to prepare some sample frames.

bvsp
Posts: 5
Joined: 2020-01-31T09:06:05-07:00
Authentication code: 1152

Re: whiteboard unique keyframes

Post by bvsp »

Also is there a way to crop the whiteboard region (large white rectangle)?
https://media.gettyimages.com/photos/sh ... d876659458

User avatar
GeeMack
Posts: 753
Joined: 2015-12-01T22:09:46-07:00
Authentication code: 1151
Location: Central Illinois, USA

Re: whiteboard unique keyframes

Post by GeeMack »

bvsp wrote:
2020-01-31T12:15:11-07:00
Also is there a way to crop the whiteboard region (large white rectangle)?
If the location of the whiteboard is constant, you can even stretch the entire whiteboard to fill the frame from corner to corner with an ImageMagick command like this...

Code: Select all

convert board.png -distort perspective "55,74 0,0 943,77 1024,0 36,514 0,683 962,512 1024,683" result.png
Edited to add: This command will perform the above stretching operation to an entire directory of PNG images...

Code: Select all

mogrify -distort perspective "55,74 0,0 943,77 1024,0 36,514 0,683 962,512 1024,683" *.png

Post Reply