WarpPlayer
I've recently become involved in a very interesting audio project with Jason McGuire. Jason is an extremely talented guitarist based in San Francisco. He hosts an online learning center -> www.flamenco-lessons.com, which I've been a part of for the last few months. The site exposes hundreds of videos, scores and detailed info about Flamenco guitar playing for registered users and provides a truly impressive learning resource. I've had several guitar teachers in my life, but few educated me as well as Jason is doing through this site... for me, another clear indication of the power of e-education.
So Jason asked me if I could think of ways to improve his site, and that's how all this audio research started. The first thing that came to my mind was having the ability to slow down the playback of his videos, maintaining the audio's frequency characteristics as much as possible, i.e. trying to maintain the audio's pitch. Slowing down the videos is great for understanding what's going on in the playing, and even to play on top of the videos, but if the pitch changes in the process, playing over it sounds simply awful, and becomes impossible... Hence, audio time-stretching would be ideal here. We are planning to produce a pretty advanced online application for the site with this feature amongst many others that will hopefully enhance the music learning experience.
What is relevant in all this to the Flash side of things, is that I am producing a media player engine which I'm calling 'WarpPlayer'. This engine should hold the core functionality of the application we are building with Jason. The player should be able to handle multimedia in an advanced manner: use dynamic audio, mix several audio tracks, be able to time-stretch audio, and sync the playback to multiple flv's or f4v's in Flash. An application based on WarpPlayer could hence play video in slow motion and time-stretch its audio, much like the way Quicktime does this with its A/V controls.
Here is a demo of the first capabilities of WarpPlayer: DEMO.
The player needs to pre-load all its assets for now (multiple sound tracks and multiple videos) before it can actually start playing, so please be patient. However, my next goal is to make it be able to start playback as soon as a small portion of its assets have been loaded.
Try playing with the small audio mixer in the left, switching video views, or the tempo controller to the right of the player.
Anyone interested in WarpPlayer becoming open-source, please leave a comment below!
Meta
Categories
Archives
- May 2010
- January 2010
- October 2009
- September 2009
- August 2009
- July 2009
- June 2009
- May 2009
- April 2009
- February 2009
- January 2009
- December 2008
- October 2008
- September 2008
Links
- Away3D
- EzeQL
- Fabrice Closier
- Infinite Turtles
- Kevin Luck
- London Flash Platform User Group
- TheTechLabs
- wu-media lab
August 9th, 2009 - 01:45
Hi, i am braziliam developer and i work on project named “Tela Brasil”
which is a school of Cinema.O site has several exercises for students to learn the basics of producing filmes.Em one of the exercises is past the basic concept of a movie soundtrack, where you can insert “layers” of sound in a movie. Still working on this project, so I am interested in “WarpPlayer” I can help with anything?
August 9th, 2009 - 02:06
Hi Gabriel,
Warp player would exceed your needs in regard to its time-stretching capabilities. From what you say, you don’t need to alter the speed of playback. You simply need to trigger sounds at specific locations of a canvas and then overlap the resulting audio composition with a muted video.
You should be able to do this without using dynamic sound at all (fp10). However, with this you would encounter some inaccuracy of a few ms in the triggering of the sounds, but then again, since you are not making musical compositions here you wouldn’t need to worry about this. If you decide to go this way, the regular as3 fp9 API should provide you with all you need.
If you do need to be accurate in the triggering of the sounds, you would need to use fp10, but the perfect library for you would be StandingWave. In its current state, it can give you all you need with perfect accuracy.
I’d love WarpPlayer to be of use to you, but given your situation I’d recommend choosing one of the 2 options above unless you wan’t to alter playback speed. That’s what WarpPlayer is meant to do.
Good luck with the project, the site looks very nice!
August 27th, 2009 - 10:41
this is very cool! I think it will definately be useful for musicans and others.
I think it would be great if you can also accelerate the tempo, keeping the same sound pitch. in fact that can be cool for all the videos on internet where you want to fast forward or slow motion which we had in tapes!
August 27th, 2009 - 15:37
Yes, it would be very useful. It can definitely be done with audio, check out Ryan’s latest experiment: http://static.ryanberdeen.com/projects/soundtouch-as3/demo/player/player_tdsi.swf
Syncing videos is a bit behind atm though. Warpplayer’s current precision can yield up to 300ms of offset between the video and the audio. I’m currently testing with FMS, maybe there is a bit more seek control on videos with it.
September 22nd, 2009 - 18:49
Hi, I work as a sound designer and music producer at Plan8 in Stockholm, Sweden. I´m not a programmer, but could the code you use in WarpPlayer be used to do perfect synch of an or several audio files to one flv file? I want to be able to put several layers of sounds and turn them on or off in precise synch to cues in the flv. I then, also would like to be able to slow down or speed up the film including the several soundlayers. I would want the sound layers / sounds to be separate from the flv som I can easily change them without having to change the video content. Is this possible to accomplish? Thanks
September 22nd, 2009 - 22:00
I can confirm that building a mixer and time stretching the total output is definitely possible. So on the audio side, yeah, go for it, flash player 10 has everything you need.
On the other hand I’m not so sure about video. With regular flash ‘pseudo’ flv streaming I found that the best sync I could get could produce offsets between the audio and the video of up to 300ms, which is a lot sometimes. I have yet to experiment with Flash Media Server with which real streaming could be done and hopefully calls to seek() would take less time to process and hence reduce the offsets. This is just my theory though, I still need to test this.