RGBD Sequences in Maya
Tuesday October 08th 2013, 9:22 am
Filed under: Kinect

This will be expanded soon. In the meantime—can you edit the camera position of an RGBD Toolkit sequence in Maya?

1. You have a virtual camera in Maya that you can place however you want, so the simple answer is: yes.

2. However! If you shot the RGBD scene with a real-life moving camera, then your material has real-life camera movement permanently baked into it. If you want to work in Maya and freely reposition a virtual camera, best to shoot the real-life scene on a tripod.

3. Unless! You want to be really clever and use match-move software (Maya and After Effects both come with trackers) to track the calibrated RGB video from the DSLR camera. (Kinect depth images are useless for that kind of tracking; way too weird-looking for match-move software to understand.) You could then create a virtual camera in Maya that matches the real-life camera you shot the scene with. Then anything you animate in Maya moves in sync with your original recording. (You’re still stuck with the original real-life camera move, however–remember that’s always permanently baked into the RGBD data.)

Comments Off on RGBD Sequences in Maya

Depth Camera Roundup
Tuesday October 01st 2013, 2:14 pm
Filed under: Kinect

Cameras that can use…

1. …either OpenNI (PrimeSense) or Microsoft drivers:

  • Microsoft Kinect 1 for Xbox (original version) … $80
  • Notes: You can’t beat the original on price and flexibility–although it has the noisiest depth data, and it always needs to be plugged into AC power. I’ve heard that it’s possible to open it up and cut the power to the little motorized platform, after which it’ll run off USB alone.

    2. …only the newest OpenNI drivers, or Microsoft drivers:

  • Microsoft Kinect 1 for Windows (revised version) … $250
  • Notes: Despite the higher price, the Kinect 1 for Windows is practically the same hardware as the Kinect 1 for Xbox. It comes with a license to sell commercial software written with the MS drivers, which isn’t really relevant to most filmmakers. Although the very latest release of the OpenNI drivers can work with the Kinect for Windows, many apps haven’t been updated to support this yet.

    3. …only OpenNI drivers:

  • Asus Xtion (all versions) … $150
  • Primesense Carmine (all versions) … $200
  • Notes: The Xtion gets cleaner depth info than the original Kinect 1 for Xbox, but I can’t recommend it for everyone because it has problems with Windows and USB3. The Carmine is supposed to be excellent, although I haven’t tried one myself yet–I will soon. Neither the Xtion nor the Carmine needs AC power; they can run off USB alone.

    4. …only Microsoft drivers:

  • Microsoft Kinect 2 (probably) … $400?
  • Notes: We don’t know much about the Kinect 2 yet, but it uses a completely different sensor than the one in the Kinect 1, Xtion, and Carmine. It’s supposed to be lower-resolution, but a lot cleaner.

    Comments Off on Depth Camera Roundup

    Image Sequence to Movie
    Wednesday June 26th 2013, 9:46 am
    Filed under: Video

    Using the free utility MPEG Streamclip, you can turn a sequence of image files into a Quicktime movie without any other software:

    Step 1. Go to Files / Open Files.

    Step 2. From the drop-down menu, choose All Files.

    Step 3. Select an image sequence and click Open. (Many image formats are supported; I recommend PNG.)

    Step 4. You may get a confusing message saying “File open error: unsupported file type.” Just click Open Anyway.

    Step 5. At this point you should be able to see your image sequence in the viewer window.

    Step 6. Now choose File / Save As.

    Step 7. Leave the drop-down menu set to MOV, a Quicktime movie file, and click Save. (If you’d like a different video format, I recommend saving out the .mov file and converting it later as a separate step.)

    Comments Off on Image Sequence to Movie

    Encoding WebM on a Mac
    Wednesday April 10th 2013, 9:12 am
    Filed under: Video

    Terry Hancock’s Lib-Ray initiative is a great alternative to Blu-ray, using high-quality HTML5 video and plain HTML/CSS and JavaScript that can play back online or from an SD card. To support all the major browsers (Firefox, Chrome, and Safari), you need to include WebM versions of your video as well as H.264—but at present the site only offers tutorials for command-line encoding tools, which are challenging for less experienced users. So I’ve created this guide to encoding WebM video using GUI utilities on OS X.

    Step 1: Download your tools. You’ll need:

    • WebM QuickTime component (free): adds WebM support to your Mac system.
    • Xiph Vorbis Quicktime component (free): adds Vorbis audio support.
    • Quicktime 7 Pro (USD $30): creates the video track. This might seem like a strange choice—but in my tests it seems to be the only GUI-based Mac tool right now that can reliably encode high-quality HD WebM video for under $100. For now, other common utilities that claim WebM support are either glitchy or have fixed low-quality settings.

    Optional extras:

    • Audacity (free): A great all-purpose audio editor that can export Vorbis audio files if you’re using separate audio tracks.
    • MKVToolnix (free): Losslessly add and delete audio and video tracks from a WebM file.

    Step 1: Install the audio and video codecs. Drop them into the /Library/Quicktime/ folder, and they should immediately become available.

    Step 2: Prepare your materials. Like H.264, WebM is a highly compressed codec, so you’ll need to start with a high-quality original to get good results. I recommend using a Quicktime movie with the PNG, ProRes, DNxHD, or PhotoJPEG codecs. Be aware that WebM doesn’t appear to support nonsquare pixels; for example, you’ll need to convert 720×480 DV video to true 4:3 or 16:9.

    Step 3: Open your video in Quicktime Pro and choose Export, then Options.

    Step 5: Video settings.
    Frame rate: You’ll have to manually match your original frame rate.
    Keyframe every: Your frame rate rounded to the nearest whole number. (If your original video is 23.976 fps, put 24).
    Data rate: Your choice—be aware this is measured in kilobytes, not bits. I use 3500KB/s for an HD original (approximately Blu-ray bitrate) and 1000KB/s (approximately DVD bitrate) for SD.
    Quality: I don’t think this affects anything, but I always set it to Best for good luck.
    One Pass/Two Pass: Two-pass encoding takes more time but gives better results. However, WebM already encodes very slowly, so if it’s not an overnight render you might want to leave this off.

    Step 6: Audio settings.
    Channels: Usually you want Stereo.
    Rate: 48Khz is standard.
    Quality: Confusingly, this is sample rate conversion quality, not compression quality. Always choose Best.
    Encoding quality: 10 is best, on a scale of 1 to 10. (Decrease if file size is important.)

    Step 7: HTML5 video lets you specify multiple sources for your video. Here’s some sample code.

       <source src=”test.mp4″ type=”video/mp4″/>
       <source src=”test.webm” type=”video/webm”/>

    You might be asking at this point—why is this odd-duck video format worth all the hassle? Well, it’s about more than just supporting Firefox—WebM was created more from legal necessity than anything else. The very popular H.264 video codec and its main physical-media distribution format, Blu-ray, are caught in a series of conflicts of interest between various giant media and technology companies. (The issues are too complex to summarize easily, but you can read more about them here.)

    These fights affect you in numerous subtle ways—licensing squabbles are why, for example, it’s so difficult to play back Blu-ray discs on a computer. H.264’s license terms allow you to give your work away for free, but they don’t let you sell it without paying royalties to the MPEG-LA, the consortium that owns the codec. To be clear, this rule isn’t currently being enforced—it’s very unlikely that anybody plans to stop you selling home-burned Blu-ray copies of your own work—but it leaves the MPEG-LA with far too much control over the livelihood of video producers. And the they could change their license terms at any time. So Google created the WebM video format to have an open-source, (theoretically) patent-unencumbered video codec to fall back on if H.264 became legally impractical to use. (Since YouTube is the world’s largest user of H.264 video, they would be extremely vulnerable to unfavorable licensing changes.) TL;DR—Including WebM as an alternative format protects your right to distribute your own work however you want. It’s an important format to be aware of, especially if you’re an individual artist selling physical copies of your work to collectors or exhibitors,

    Blender on a Mac Laptop
    Thursday January 24th 2013, 3:20 pm
    Filed under: Blender

    Blender has a reputation for being difficult to navigate, but you can get around just fine with Mac trackpad gestures—you just need to set things up properly:

    Step 1.
    Go to File / User Preferences.

    Step 2.
    Go to the Input tab.

    Step 3.
    Make sure the Presets drop-down menu is set to Blender. (An option for Maya controls was a nice idea…but it’s Maya’s older control scheme that requires a middle mouse button. So it’s not going to help us.)

    Step 4.
    Check Emulate 3 Button Mouse.

    Step 5.
    Choose Save As Default.

    Step 6.
    You should now be able to navigate using your trackpad:

    Pan: Two-finger scroll
    Rotate: Alt + two-finger scroll
    Zoom: Two-finger pinch

    Step 7.
    If, like me, you don’t care for the default Blender keyboard shortcuts, follow these instructions.

    Thursday August 30th 2012, 8:27 am
    Filed under: Kinect

    Victoria Nece and I have released a compiled app and UI panel for getting Kinect mocap data into After Effects—it’s called KinectToPin. For a step-by-step guide, read through this tutorial.

    Comments Off on KinectToPin

    Multi Medias
    Friday March 23rd 2012, 9:26 am
    Filed under: Concepts

    (Hit counts from search performed 23 mar 12.)

    Mixed media (1950s?, fine art, 35M hits): Describes a work that mixes two or more visual media, often restricted to the disciplines of painting and sculpture.

    Intermedia (1960s, fine art, 25M hits): Describes a work that mixes any two or more artistic disciplines usually recognized as separate. Popularized by the Fluxus group. (Coined by Dick Higgins.)

    Hypermedia (1960s, computer science, 20M hits): Describes an organized collection of media files that the user can move between at will. Wikipedia is a hypermedia system. (Coined by Ted Nelson.)

    Multimedia (1.2B hits)
    1. (1960s, fine art): Describes a work that mixes any two or more media; largely replaced in current usage by “intermedia.” (Possibly coined by Bob Goldstein.)

    2. (1990s, computer science): Describes a device that can display multiple types of media–sound, video, text, images, etc.

    3. (1990s, popular): Describes a work in any medium either created or experienced using such a device; largely replaced in current usage by “new media.”

    New media (1990s, media theory, 69M hits): Describes works which are both created and experienced using a computer.

    Transmedia (1990s, media theory, 3M hits): Describes a narrative dispersed across multiple works in multiple media, each of which contains only a portion of the complete story. A long-running franchise like “Star Wars” can be considered one epic transmedia work. (Coined by Marsha Kinder.)

    Rich media (2000s?, design, 9M hits): Describes works distributed via the internet that make use of sound and video, as opposed to images and text.

    Comments Off on Multi Medias

    3D Depth Cues
    Sunday October 30th 2011, 11:31 am
    Filed under: Concepts

    1. Motion parallax: distant things move slower; close things move faster.
    2. Depth in motion: something coming closer to you gets bigger.
    3. Perspective: parallel lines converge as they move away from you.
    4. Familiar size: evaluating the position of an object whose size is known.
    5. Relative size: comparing the position of an unknown object to one whose size is known.
    6. Color and contrast: all else being equal, reds appear closer than blues, and high-contrast colors appear closer than low-contrast colors.
    7. Accommodation: whether the eye is focused on a near or far object, as reported by our eye muscles.
    8. Occlusion: an object blocking something else is assumed to be in front of it.
    9. Depth of field: blurrier objects are assumed to be further away.

    10. Stereopsis: the difference between the views of the left and right eye.
    11. Convergence: the angle from our eyes to the object viewed, as reported by our eye muscles (for objects closer than ~10m).

    Comments Off on 3D Depth Cues

    OBJ Batch Export
    Saturday October 22nd 2011, 7:34 am
    Filed under: Maya

    Here’s how to export a sequence of frames from Maya as individual OBJ files:

    Step 1. Download this OBJ Exporter MEL script.

    Step 2. In Maya, go to Window / Settings/Preferences / Shelf Editor to add the script as a button.

    Step 3. Click the New Item icon to create a new blank button.

    Step 4. Click the Icon Name icon to add a custom image for your button (included with the download).

    Step 5. Go to the Command tab and copy-paste the script in. Make sure the radio button is set to MEL, the language the script is written in.

    Step 6. Click Save All Shelves.

    Step 7. The new button should now appear in your shelf. Click it…

    Step 8. …to open the OBJ batch export panel.

    An OBJ sequence is a great way to do replacement animation with 3D printouts:

    Comments Off on OBJ Batch Export

    Format Suggestions
    Tuesday October 04th 2011, 7:06 pm
    Filed under: Concepts

    1. Archival master: This is the copy you file away forever, in the highest quality available, and in an “open” format that you can be reasonably sure will be available many years into the future. (That is, it’s open-source, or the patent’s expired). Usually a format meeting these requirements can’t be played back easily, or at all–it’s just for storage.


    • picture:  1920×1080 @ 12fps, 23.976fps, or 24fps.
    • sound:   48KHz 24-bit, stereo or discrete 5.1
    • codecs:  EXR image sequence (32bpc) or PNG image sequence (16bpc or 8bpc); uncompressed AIFF or WAV files.

    2. Submaster: This is the copy you make from the master and work with day-to-day. It’s slightly smaller and lower-quality than the master, and usually in a proprietary format that might be gone in five years, but works great for now. (A high-quality HD video that still plays back adequately on an older computer is a relatively recent invention.) You can also use the submaster to make DVD and Blu-ray discs if you need to.


    • picture:  1920×1080 @ 23.976fps or 24fps.
    • sound:   48KHz 16-bit, stereo or discrete 5.1
    • codecs:  Quicktime with Apple ProRes, GoPro Cineform, Avid DNxHR, or PhotoJPEG codec; PCM audio.
    • note: Quicktime DV is still a popular choice for SD material, but be aware that —unlike DVD discs—the DV tape standard only supported NTSC and PAL frame rates. 24p DV tape hardware used hacky workarounds specific to the manufacturer, so if you’re working with some legacy material I would transcode it all into a modern codec.

    3. Distribution copies: These are the copies you make from the submaster and hand out to your audience. They’re much smaller and lower-quality than the submaster, but can be easily passed around and viewed on many devices.


    • picture:  1920×1080@ 23.976fps or 24fps.
    • sound:   48KHz 16-bit, stereo or discrete 5.1.
    • codecs:  H.264 MP4 video; AAC or AC3 audio.
    • note:H.264 is currently the established standard for distribution, but IP-rights squabbles in the past (now largely resolved) kept it out of certain free-software projects, like the Chromium browser. This doesn’t affect you in most common situations, but it may be worth making a distribution copy in the fully open WebM format as a hedge.
    Comments Off on Format Suggestions