Segmenter Component (mpf-segmenter)

Published June 1, 2009, 4:04 pm | by stuffs

I just committed a first version of a stream segmenter component: mpf-segmenter.  This component works together with a time-of-interest component (mpf-toi) to chop a stream up into segments of interest. Both mpf-toi and mpf-segmenter are in the public SVN tree under the MPF/components directory, and require a current build of GRDF which implements node comparison functions for uri and anon nodes.

Use-cases (tests):

1. simpletest

# Take a media stream, tee it into time-of-interest analyzer and the mpf-segmenter, then take the
# toi metadata into the segmenter so it can segment the media stream (fake in this case).  Finally,
# sink the media stream.  The time-of-interest is 2-6 seconds in a 10 second stream.

mkdir out

$1 gst-launch $2 fakesrc sizetype=2 sizemax=10 filltype=5 num-buffers=10 datarate=10 ! tee name=t \
  t. ! mpf-toi start=2 duration=4 $3 name=toi ! seg. \
  t. ! mpf-segmenter name=seg $4 ! mpf-multifilesink location="out/simpletest-out-%015lld"

2. multi-segments

# Take a media stream, tee it into time-of-interest analyzer and the mpf-segmenter, then take the
# toi metadata into the segmenter so it can segment the media stream (fake in this case).  Finally,
# sink the media stream.  The time-of-interest is 2-6 seconds, repeating at 10 seconds in a 20 second
# stream, which should yield two segments (i.e. sets of files with contiguous timestamps).

mkdir out

$1 gst-launch $2 fakesrc sizetype=2 sizemax=10 filltype=5 num-buffers=20 datarate=10 ! tee name=t \
  t. ! mpf-toi start=2 interval=10 duration=4 $3 name=toi ! seg. \
  t. ! mpf-segmenter name=seg $4 ! mpf-multifilesink location="out/multi-segments-out-%015lld"

3. red-baron

# Segment an audio stream into 2 second chunks, skipping by 10 seconds.  To play the output file:
# gst-launch filesrc location=out/red-baron-out.mp3 ! decodebin ! audioconvert ! osssink

mkdir out

$1 gst-launch $2 filesrc location=data/RedBaron.wmv-0s-END.mp3 ! decodebin ! tee name=t \
  t. ! mpf-toi interval=10 duration=2 $3 name=toi ! seg. \
  t. ! mpf-segmenter name=seg $4 ! audioconvert ! lame ! filesink location="out/red-baron-out.mp3"

4. video-segments-silent

# Segment a video stream into 2 second chunks, skipping by 10 seconds.  To play the output file:
# gst-launch filesrc location=out/red-baron-out.mp3 ! decodebin ! ffmpegcolorspace ! ximagesink

mkdir out

$1 gst-launch $2 filesrc location=data/video_02_silent.mp4 ! decodebin name=dec dec. ! tee name=t \
  t. ! mpf-toi interval=4 duration=2 $3 name=toi ! seg.metadata \
  t. ! mpf-segmenter $4 name=seg ! ffmpegcolorspace ! ximagesink

At present, the segmenter cannot handle audio+video streams, it crashes with a null buffer return from mpf_pull_buffer.  I"m working on this.

Note that all scripts can be run with parametrization. For example:

./simpletest "ddd --args" "" "verbose=2" "verbose=2"

brings the simple-test pipeline up in the DDD debugger (you could use simple gdb), and turns verbose on for mpf-toi and mpf-segmenter.

The metadata flowing from mpf-toi to mpf-segmenter is currently a value ranging between 0 and 100, and the exact form will be changing as namespaces are added.   The mpf-segmenter is set to pass data whenever the toi interest exceeds a threshold of 50.