WarpPlayer
I've recently become involved in a very interesting audio project with Jason McGuire. Jason is an extremely talented guitarist based in San Francisco. He hosts an online learning center -> www.flamenco-lessons.com, which I've been a part of for the last few months. The site exposes hundreds of videos, scores and detailed info about Flamenco guitar playing for registered users and provides a truly impressive learning resource. I've had several guitar teachers in my life, but few educated me as well as Jason is doing through this site... for me, another clear indication of the power of e-education.
So Jason asked me if I could think of ways to improve his site, and that's how all this audio research started. The first thing that came to my mind was having the ability to slow down the playback of his videos, maintaining the audio's frequency characteristics as much as possible, i.e. trying to maintain the audio's pitch. Slowing down the videos is great for understanding what's going on in the playing, and even to play on top of the videos, but if the pitch changes in the process, playing over it sounds simply awful, and becomes impossible... Hence, audio time-stretching would be ideal here. We are planning to produce a pretty advanced online application for the site with this feature amongst many others that will hopefully enhance the music learning experience.
What is relevant in all this to the Flash side of things, is that I am producing a media player engine which I'm calling 'WarpPlayer'. This engine should hold the core functionality of the application we are building with Jason. The player should be able to handle multimedia in an advanced manner: use dynamic audio, mix several audio tracks, be able to time-stretch audio, and sync the playback to multiple flv's or f4v's in Flash. An application based on WarpPlayer could hence play video in slow motion and time-stretch its audio, much like the way Quicktime does this with its A/V controls.
Here is a demo of the first capabilities of WarpPlayer: DEMO.
The player needs to pre-load all its assets for now (multiple sound tracks and multiple videos) before it can actually start playing, so please be patient. However, my next goal is to make it be able to start playback as soon as a small portion of its assets have been loaded.
Try playing with the small audio mixer in the left, switching video views, or the tempo controller to the right of the player.
Anyone interested in WarpPlayer becoming open-source, please leave a comment below!
Doritos iD3
RehabStudio and AwayMedia have produced a pretty amazing FWA winner advergame for Doritos' latest iD3 campaign. My involvement in the project was developing the adventure's 3D mini games with Away3D. Rehab has done a superb job with the interface and the video content on the site is simply 'cinematic'... Click the image to try out the adventure!
HINT: There's a cheat on one of the mini games. Hit up, up, down, down, left, right, left, right, b, a, enter, while near the forklift to drive it =)
[UPDATE] If you don't have a Doritos pack code, you can get free non-winning codes here: http://id3.doritos.co.uk/codeclaim.php
Pitch shifting and time stretching in AS3
Audio time stretching is the process of slowing down or speeding up a piece of audio without altering its pitch. Its complement is pitch shifting, which is the process of altering the pitch of a piece of audio without altering its tempo.
This transformations are not as straight forward as they seem to be. This is primarily because any transformations applied to an audio signal in the time domain usually affect its frequency domain, and vice versa. I've been researching this a lot lately and I must admit I bumped into a pretty big and mean realm. If an attempt to preserve audio quality is made, very complex algorithms (made by some very smart people) start showing up. Different DSP (digital signal processing) techniques exist for this purpose such as the Phase Vocoder, TDHS, WSOLA, PSOLA, etc... you can read more about the topic on wikipedia here.
So, what I did was grabbed Ryan Berdeen's port of the C Sountouch library to AS3 (soundtouchAS3) and modified it to act as a StretchFilter in Joe Berkovitz's StandingWave2 (standingwave) dynamic audio library for flash 10. After adding this filter to standingwave, it was just a question of combining it with the lib's ResamplingFilter, and voila, pitch shifting.
DEMO: here.
****WARNING****: This is quite cpu intensive (the implementation is far from optimized), so be prepared to turn the volume down! Also, please be patient with the sound loading, and sorry for the crappy interface! This is just an experiment. Avoid touching the controls before a sound is loaded (:P). When the sound loads, try altering the pitch or the tempo, one at a time.
Clearly, my adaptation of the time stretching algorithm needs to be optimized and perhaps even ported back to C! (for use with Alchemy) but the main purpose of this experiment is just to show that time stretching and pitch shifting in AS3 is possible... which is pretty good news for us Flash geeks, right?
This should have a few uses. I can think of a couple: Dynamically altering the tempo or pitch of media the way Quicktime player does with the A/V controls (would be pretty good for tutorials, specially music tutorials), being able to warp audio sources in some sort of online audio editor, etc.
Meta
Categories
Archives
- May 2010
- January 2010
- October 2009
- September 2009
- August 2009
- July 2009
- June 2009
- May 2009
- April 2009
- February 2009
- January 2009
- December 2008
- October 2008
- September 2008
Links
- Away3D
- EzeQL
- Fabrice Closier
- Infinite Turtles
- Kevin Luck
- London Flash Platform User Group
- TheTechLabs
- wu-media lab
