Doritos iD3
RehabStudio and AwayMedia have produced a pretty amazing FWA winner advergame for Doritos' latest iD3 campaign. My involvement in the project was developing the adventure's 3D mini games with Away3D. Rehab has done a superb job with the interface and the video content on the site is simply 'cinematic'... Click the image to try out the adventure!
HINT: There's a cheat on one of the mini games. Hit up, up, down, down, left, right, left, right, b, a, enter, while near the forklift to drive it =)
[UPDATE] If you don't have a Doritos pack code, you can get free non-winning codes here: http://id3.doritos.co.uk/codeclaim.php
Pitch shifting and time stretching in AS3
Audio time stretching is the process of slowing down or speeding up a piece of audio without altering its pitch. Its complement is pitch shifting, which is the process of altering the pitch of a piece of audio without altering its tempo.
This transformations are not as straight forward as they seem to be. This is primarily because any transformations applied to an audio signal in the time domain usually affect its frequency domain, and vice versa. I've been researching this a lot lately and I must admit I bumped into a pretty big and mean realm. If an attempt to preserve audio quality is made, very complex algorithms (made by some very smart people) start showing up. Different DSP (digital signal processing) techniques exist for this purpose such as the Phase Vocoder, TDHS, WSOLA, PSOLA, etc... you can read more about the topic on wikipedia here.
So, what I did was grabbed Ryan Berdeen's port of the C Sountouch library to AS3 (soundtouchAS3) and modified it to act as a StretchFilter in Joe Berkovitz's StandingWave2 (standingwave) dynamic audio library for flash 10. After adding this filter to standingwave, it was just a question of combining it with the lib's ResamplingFilter, and voila, pitch shifting.
DEMO: here.
****WARNING****: This is quite cpu intensive (the implementation is far from optimized), so be prepared to turn the volume down! Also, please be patient with the sound loading, and sorry for the crappy interface! This is just an experiment. Avoid touching the controls before a sound is loaded (:P). When the sound loads, try altering the pitch or the tempo, one at a time.
Clearly, my adaptation of the time stretching algorithm needs to be optimized and perhaps even ported back to C! (for use with Alchemy) but the main purpose of this experiment is just to show that time stretching and pitch shifting in AS3 is possible... which is pretty good news for us Flash geeks, right?
This should have a few uses. I can think of a couple: Dynamically altering the tempo or pitch of media the way Quicktime player does with the A/V controls (would be pretty good for tutorials, specially music tutorials), being able to warp audio sources in some sort of online audio editor, etc.
Understanding dynamic sound in Flash
I made a little test app in order to help me understand how dynamic sound in Flash works. I had done a few of the basic tutorials/experiments and I kind of got the idea, but quite a lot of mystery still remained in the topic... While doing this experiment though, I feel like I have a much more thorough understanding of the way dynamic sound works in Flash and so, I decided to share it, 'cause I would have loved to find something like this on someone elses blog. Besides, I feel like the more people are familiar with this the better because this feature of FP10 is simple awesome!
So, how does dynamic sound work? The new method in Sound called "extract" can read audio data from any position of a Sound object. It extracts the data into a ByteArray, that is, into a bunch of 0s and 1s. But the hardest thing for me to understand was: What is the nature of this data? How does it describe a sound?
Well, byte arrays are highly efficient arrays that contain boolean values, or bits. These 0s and 1s are packed in groups of 8, each being a byte. So byteArray.position = 256 would not place the array pointer at bit 256, it would place it at byte 256, that is, at bit 2048. Now, the extract method populates the byte array with a bunch of bytes, but in order to make sense out of the bytes, we must visualize them as groups of 8 bytes (64 bits). Grouped in this way, each group is what we call a sample in audio...
Real life sound is nothing more than oscillations or vibrations that travel through air. The amplitude of these oscillations correspond to volume and the cycle or wavelength of these oscillations correspond to what we interpret as pitch. The way digital sound works is to "sample" the instantaneous amplitude of these vibrations at a very fast rate (say 44100Hz, that is 44100 samples each second). Such amplitude readings can be stored in 32 bit "float" values. Flash works at 44100Hz, stereo, so the first 4 bytes of each sample correspond to an instantaneous value in the left channel and the next 4 to the amplitude of the right channel. 32 bits per channel. So if we use the extract method, reposition the byte array in 0, we can use the readFloat method of ByteArray in pairs, and hence extract these amplitude values of the entire sound, one by one. Doing this, its crucial to keep in mind that each time you read a part of the byte array, the position marker is shifted to the end of what you've just read. Its just the way byte arrays work, differing from regular arrays.
In this manner, we can read large chunks of samples and give them to the data object in the SampleDataEvent. Doing so, we are delivering a chunk of audio to the computer's audio card so it can make some noise. The beauty in this is that we can read and understand the sound data, and before giving it to the sound card, we can process the audio with complex filters and DSP analysis, use the information for visual display, etc... This is why it is handled in chunks of audio, so we can be able to process this sound in real time.
Its important to note that when we extract a chunk from the Sound object, we don't know if (imagine that we are arriving to the end of a sound) we are going to be able to get the amount of samples we asked for, so the extract method returns a value, indicating how many samples it was able to extract.
So enough talking... see it yourself in the demo by clicking on the image above (view source enabled). In this experiment, I am not doing anything to the original sounds, just using the info extracted each cycle to visualize whats going on. The boxes plot the amplitude of the samples againts time, and from the pixelated look of them, you can easily grasp how many samples these mp3's have... A lot of them. The upper box shows the waveform of the entire sound clip (note that it is not a frequency spectrum), and the lower box shows the instantaneous waveform of the chunk that has just been extracted from the Sound object and been delivered to the sound card. You can change the playback speed or drag the playback head to see the process in slow motion. You can also change the buffer size, which is nothing more that the size of the chunks (in samples) that are processed at a time on each cycle. And, you can change the sounds too, just because waveforms are beautiful... be patient on the loading though, my server is sloooooow...
Processing sounds can be quite cpu intensive, but luckily we have pixel bender and alchemy to give us a little more juice in this area. I'd like to post more about this topic in the future, its just too interesting. Perhaps a spectrum analyzer, pitch shifting, time scale modification, etc... The possibilities are endless. Just look at what guys like Andre Michelle are doing!
Extruding TextField3D
We've added a simple class that provides extrusions for TextField3D objects. Right click on the demo to see the source and learn how to use the feature.
UPDATE: Properties "textColor" and "textAlpha" have been replaced by "material". Since TextField3D supports various material types now, these two properties are not applicable any more.
Away3D currently supports all the basic color materials on irregular shapes like these. You can use WireColorMaterial, ColorMaterial, WireframeMaterial and ShadingColorMaterial on vector text and vector graphics coming from Flash.
There is still a lot of work to do, but we're getting there!
Importing graphics from Flash to Away3D
In a very similar way we used the wumedia package to extract fonts from Flash, we can use it to extract vector graphics in general from Flash. This was possible before in Away3D using VectorSwf.as, as shown on a previous post, but the work flow that I will present now is so much simpler, that we've decided to deprecate VectorSwf.as and replace it with the much more powerful and easier to use Swf.as
Click on the image to see what I'm talking about (right-click view source enabled). The snowman was designed in Flash, and a swf was exported containing this graphic and other vector assets that may want to be used in 3D. After that, the assets swf is loaded using the following code, which is the same way many other external assets are loaded with Away3D, such as Collada, 3ds, Obj, etc:
_loader = Swf.load("assets/vectorassets.swf", {libraryClips:["Snowman"], scaling:2});
"libraryClips" is an array containing all the linkage names of the clips that want to be extracted. It is important to set these linkages in Flash in order to be able to identify the clips from the main app. The parameter is optional however, if not supplied, Swf.as will attempt to extract the graphics from the stage. For this to be possible however, the instances on the stage must be set as graphics types.
Another thing to take into consideration is the flattening of the clips. As one of our current goals is to be able to extrude these clips, Away3D treats flattened graphics differently from graphics that are grouped, or on layers, and non flattened graphics might produce sorting problems at this stage. In order to correctly flatten a vector graphic in Flash (as for instance a drawing imported from Illustrator), the way I recommend it is:
- Alternate "break apart" with "distribute to layers" until no grouped elements are left.
- Select all the resulting shapes and go to Modify -> Shape -> Convert Lines to Fills.
- Copy the content of all the produced layers and paste it into a new clip.
This simple process has ensured me so far to be able to migrate any graphic from Flash to 3D properly. Even pretty complex ones copy-pasted from Illustrator.
Sometimes its good to keep layers however. Imagine a window component that you wish to take to 3D. The window could have a title, a title bar, a frame and a background. And now imagine that you want this window to remain in layers as opposed to being flat (eg, the title bar above the background, title above the title bar). What you do is skip the flattening and use:
{libraryClips:["Snowman"], scaling:2, perspectiveOffset:-100, perspectiveFocus:2000}
in the init object passed to the constructor. This example would separate the layers by 100 pixels backwards, and if viewed exactly 2000 pixels from the front of the container in which the graphics were imported to by Swf.as, you will see exactly what you saw in 2D! With perspective correction of course. As each element is drawn back, it is scaled up so that the overall image doesn't loose proportion in each of its elements. If you move from this focus point however, you should become aware of the layering and the offsets between the graphics.
I hope someone finds usefulness in all this. And please, do provide feedback if you've got any ideas or requests!
TextField3D Tutorial
So, what's new with dynamic 3D text in Away3D now? Not much from the end-user point of view, but the internals have changed dramatically from previous implementations. What's most important is that TextField3D is now a permanent class in the engine. We no longer have to use the textfields branch in order to use vector features. The branch was experimental, is now deprecated, and all the investigated features are now being assimilated in the main trunk.
Dynamic textfields are now using Guojian Wu's powerful swf parser. You can see this magnificent tool and more of his work at wu-media.com. The parser allows us to use any font installed in the system, and provides many text wrapping features such as leading, kerning, alignment, etc.
There are many ways to use it. The basic principle is that Wu's parser identifyes a font's byte data in an swf and extracts its vector information. Font byte data can be in the same swf as the main application, or in a different asset swf. The way I recommend to use it is as follows:
1) Create a new FLA file with Flash, create a dynamic textfield for each font on stage, and on each use the properties panel to embed the characters you will want to be using. Export the swf as, for example, fonts.swf.
2) In another actionscript project, use an URLLoader to load the fonts.swf as binary data.
3) Once the data is loaded, use "VectorText.extractFont(_loader.data)" so that the parser can process the fonts. NOTE: In flex, you can also embed the fonts.swf file, and pass the bytes to extractFont() as "new FontBytes()".
4) You can now create TextField3D instances as:
var textfield:TextField3D = new TextField3D("Arial", {textAlpha:1, textColor:0x000000, text:"TextField3D", size:200, leading:0, kerning:0, textWidth:2000, align:"TL"});
_scene.addChild(textfield);
UPDATE: Properties "textColor" and "textAlpha" have been replaced by "material". Since TextField3D supports various material types now, these two properties are not applicable any more.
Click on the image above to see a demo, and then right-click on the demo to view the source.
What's next? In the next few days we will continue to assimilate features from the textfields branch into Away3D's main trunk. Amongst these coming features are vector text extrusion, support for advanced shading materials on vector primitives like text, importing of sprites from Flash with wu's parser in a very similar way as using text in this tutorial, tessellation tools for vector primitives, extrusion of sprites imported from Flash, etc.
I'll make sure to post any updates on this progress. On the mean time, I hope you enjoy these basic text features, and don't hesitate to post any suggestions here, or in the Away3D user group!
3D LensFlare
A simple lens flare effect has been added to Away3D's main trunk. Click on the image above to see the demo (view source enabled).
Please feedback any ideas, adjustments, etc that would make it more realistic!
Morgan Stanley Matrix
Working with Morgan Stanley and a small team of superb designers from Adobe Profesional Services on this microsite was one of the best work experiences I've had. Have a look: www.morganstanley.com/matrixinfo
The site was built using Away3D + Awaybuilder, and it uses simple yet slick camera motion tweens to fly around a space crammed with vector content. Such content was provided with a friendly workflow using the JSFL tools from my previous posts. Additionally, a pretty interesting class was used to "perspectivate" the vector glyfs so that most of the text is only visible correctly from a given position in 3D space. The site is a pretty good example of the crisp look of vectors in 3D, and makes me think of all the possibilities we have with this sort of tools.
VectorSwf tutorial
[UPDATE] VectorSwf.as is now deprecated. The replacement is Swf.as, which provides a much simpler workflow. See upcoming post!
This tutorial attempts to explain the features described in the previous post, so you can use them, and hopefully apply them for something more useful than this demo!
Setting up the project.
--------------------------------------------------------------------------------
1 - Create a new Actionscript project in Flex. Delete the default class created by Flex and paste these 2 classes into the project src folder. Set Main.as as the Default Application.
2 - Link your project to the latest Away3D trunk. You can either do this by setting up a separate library project and linking the AS project via properties/ActionScript Build Path/Library path or simply pasting a copy of the away3d lib in the src folder of your AS project.
3 - In the project properties/ActionsScript Compiler, add "-use-network=false" in the Additional Compiler arguments field to avoid security sandbox problems.
4 - In the src folder of your project create a new flash file named Assets.fla and allow it to publish at the same level Assets.swf.
Prepare your assets in Flash:
--------------------------------------------------------------------------------
5 - Create a new clip and, in it, draw some vector graphics. Make sure not to use strokes because they are not supported yet. Sorry! In my case I downloaded an .eps from www.freevectors.net and copy pasted the content into the clip.
6 - We now need to run a JSFL command from Flash in order to inject its vector data into a variable in the clip's timeline. For this download the jsflaway lib and place it in your flash commands folder. In my case it is "Users/yourUserName/Library/ApplicationSupport/Adobe/FlashCS3/en/Configuration/Commands/"... then while in the clip's edit mode (in its stage) in Flash, go to Commands, run Command and go to the jsflaway lib you just copied to your HD and select VectorDataInjector.jsfl in the components package of the lib. You will see that this should create a new layer in your clip called "vectorData".
7 - Drag an instance of your clip to the main stage of your Assets.fla and publish it.
8 - Repeat steps 5 to 7 to add more elements. As long as an element is on the stage, and the clip contains the vectorData layer, Away3D should be able to port it to 3D.
Away3D code.
--------------------------------------------------------------------------------
9 - You can tweak some settings in the src:
var vectors:ObjectContainer3D = VectorSwf.load("Assets.swf", {scaling:1, zOffset:100});
_scene.addChild(vectors);
Scaling speaks for itself, and zOffset is the z spacing applied between each clip in the main stage of the Assets fla.
SOURCE: http://www.lidev.com.ar/demos/textfieldsmerge/vectorswf/v5/tutorial_vectorSwf.zip
I hope you can find useful applications for this feature. You can use it to show vector interfaces in 3D, text, and practically anything made out of vectors. Steps 5 to 8 should be greatly simplifyed soon thanks to feedback I already recieved from you guys. I'll let you know when I have some updates. Enjoy!
Away3D and Vectors
[UPDATE] VectorSwf.as is now deprecated. The replacement is Swf.as, which provides a much simpler workflow. See upcoming post!
Long time no post. Sorry for that. The good news is that I've been extremely busy, specially with the way Away3D handles vector graphics, and this is the main reason why I haven't brought news any earlier. Let me get you updated now, tell you what's been going on, what tools I'm currently developing and what you can expect the future of vectors in Away3D to be.
First of all, development on the textfields branch has stopped. You can still use it for any of the functionalities described in previous posts, but the thing is that the branch was simply an experimental space intended for 3D vector graphics research and we've now moved on to the official development on the main trunk, where we've taken a completely different approach. The core concept is that now, a regular face can handle more than 3 sides, and these sides can be curved. It's a pretty radical approach since the basis of 3D is triangles. These triangles can now mutate and take any shape that can be described with bezier curves.
Another significant change is the priority given to the Flash IDE as a source for vector graphic elements. Although development on external ttf loading, dynamic font parsing, svg compatibility, etc, is still planned to be supported, the focus is redirected to using Flash for managing text and other vector graphics and then exporting them for a 3D project. This is done by using JSFL to extract vector data in the Flash IDE and then export it in order to use it in a 3D project. In my personal experience, I've found that until Adobe decides to release the vector data that conforms sprites, this is the most practical way to work with such data in order to project it onto the 3rd dimension.
The above demo (click on first image) was achieved by downloading a free eps (Illustrator) file from the web, copy pasting it in Flash, running a JSFL script in Flash in order to inject the vector data in an accessible way into an assets swf and then reaching such data from the 3D project with a couple of lines of code:
_vectorObj = VectorSwf.load("assets/vectorScene.swf", {scaling:1, zOffset:100});
_scene.addChild(_vectorObj);
As you can see, the compiled .swf is treated by VectorSwf as Collada treats .dae files, so these features should make it really easy to work with vector assets in 3D and provide a series of practical means to obtain crisp 3D graphics. I promise to include a tutorial for the above demo in my next post so you can start playing with it and providing me some feedback in order to continue the development of these features.
On the mean time, if you feel like exploring the demo, here is what you need to get it running:
DEMO: http://www.lidev.com.ar/demos/textfieldsmerge/vectorswf/v4/
SOURCE: http://www.lidev.com.ar/demos/textfieldsmerge/vectorswf/v4/test_vectorSwfLoader.zip (Link it to the latest Away3D source).
JSFLAWAY: http://www.lidev.com.ar/demos/textfieldsmerge/vectorswf/v4/jsflaway.zip (Run VectorDataInjector.jsfl in order to inject the vectorData array into a clip).
Meta
Categories
Archives
- May 2010
- January 2010
- October 2009
- September 2009
- August 2009
- July 2009
- June 2009
- May 2009
- April 2009
- February 2009
- January 2009
- December 2008
- October 2008
- September 2008
Links
- Away3D
- EzeQL
- Fabrice Closier
- Infinite Turtles
- Kevin Luck
- London Flash Platform User Group
- TheTechLabs
- wu-media lab
