Babylon 5 2×01 Effects Update Supplemental- The Three-Edged Sword

The surprise Babylon 5 HD remaster came out on Blu-Ray recently. Something I was curious about was how difficult it would be to combine the 4×3 Blu-Rays with the 16×9 DVDs to get a 16×9 pseudo-HD version of the show, with the edges upscaled but the center using real film.

I did a quick test on a frame from “Points of Departure,” and the answer is “not as easy as I hoped.” I’ve tried this technique before, matching and blending the same film from two different releases (and, apparently, two different digital scans), and I’ve also run into this problem of the film not scanning perfectly flat, and having some amount of distortion. I’d have to do further research to see if there are tools to address this that I have access to,1It feels like a combination of technologies should be able to solve this; I’ve got two slightly different sources of the same ground-truth. The HD frame shows exactly how 3/4th of the SD frame should look after it’s upscaled, so it should be possible to have a computer program compare the two, adjust the color, size, rotation, and proportion of the SD frame to match the HD one as closely as possible, apply some ML upscaling (ideally using the very shot that’s being upscaled as training data so the computer can extrapolate the lost detail at the sides of the frame from the existing detail in the middle), and blend the two sources. I know all these features exist independently, but I have no idea how to get them all into one tool. There might also be some complications with the DVDs needing to be detelecined and having scanline issues in the raw data. so this is just a quick test and comparison using Photoshop. The vertical framing of the DVD and Blu-Ray don’t match exactly, and the DVD’s colors are a little more contrasty, so I had to make some adjustments to match better.

First off, the test image merging the two sources. I upscaled the DVD frame using an AI tool, color-corrected it as described above, and added some grain, while softening the border between the HD frame and the widescreen one:

It could be worse!

Continue reading

References
1 It feels like a combination of technologies should be able to solve this; I’ve got two slightly different sources of the same ground-truth. The HD frame shows exactly how 3/4th of the SD frame should look after it’s upscaled, so it should be possible to have a computer program compare the two, adjust the color, size, rotation, and proportion of the SD frame to match the HD one as closely as possible, apply some ML upscaling (ideally using the very shot that’s being upscaled as training data so the computer can extrapolate the lost detail at the sides of the frame from the existing detail in the middle), and blend the two sources. I know all these features exist independently, but I have no idea how to get them all into one tool. There might also be some complications with the DVDs needing to be detelecined and having scanline issues in the raw data.

Babylon 5 2×01 Effects Update Supplemental- Their Horses Were On Fire

With the Audio Guide to Babylon 5 getting into the third season of the show, and RedTed’s Babylon 5 model coming along nicely, I’ve been feeling more excitement about my old B5 effects project, which has been safely tucked in the back of my mind for the past few years. I think I’d like to take care of some more of the prerequisites before I jump back into making shots, not in the least because I’d also like to finish my current Star Trek project without introducing one of my trademark delays. I also think it’d be prudent for me to act as if this isn’t going to be a one-episode wonder, and take my time in laying the groundwork for continuing this project indefinitely.

Luckily, in my last post, I promised an investigation into the three depictions of the Battle of the Line in Babylon 5. I’d like to make all the depictions of the battle as consistent as possible, so I wanted to catalogue what happened, and which redone shots corresponded with which earlier shots. Let’s start with a chart of my findings, and then there will be some further discussion. I’ve listed the shots in chronological order.

Continue reading

Babylon 5 2×01 Effects Update Part 004- Steamed, Not Boiled

Today’s update is a fairly straightforward sequence. We’ve now reached Act 2 of the episode, which begins with one of the common season 1 establishing shots of the station.

The stock shots of the station were something I wondered about earlier in this project. On the one hand, part of the philosophical basis for doing this is to give the show a more unified appearance, rather than the variations from shifting styles and evolving technology during the original run. On the other hand, I would like it if each chapter of the show had it’s own style (more than just how many and what kind of spaceships are loitering around Babylon 5). This didn’t quite occur in the original run, with very early stock footage being used in later episodes where the look of the show had evolved into something totally different, so what I’m shooting for can be described as stylistic evolution over the five seasons, but technical consistency for the run of the show. Another concern was the ever-present question of when I want to make a direct recreation of the original shot, and when I think I have my own idea for how to set it up. It’s hard to think of a “better” way to do any given generic, interchangeable establishing shot, as opposed to something with a more specific context.

What I think I’ll end up doing is to try to preserve the “feel” of the original establishing shots, while recreating them in my own idiom. For instance, a lot of the establishing shots in the first season of B5, including this one, either started from came to a complete stop, with a very spline-y acceleration or deceleration that screams “early computer graphics.” Eliminating that, along with framing it from the start in 16×9, makes the shot feel a lot more comfortable and cinematic than the version on the DVD.

Assuming this project continues on, I’ll probably make adjusted versions of early shots when they pop up in later seasons to fit in better with those episodes. A good example of this is one of my favorite touches from the new Battlestar Galactica. There was one particular establishing shot of the ship which was used throughout the series, and in later appearances, as the Galactica became more beat-up, new versions of the exact same shot appeared with the additional weathering. It was a fun twist on the concept of stock footage, and something I’d like to do myself.

The other segment in this update is from (the first half of) Sheridan’s good-luck speech in the Observation Dome. There are two shots out the window, one of which had a pivot and a zoom, and the other which was locked off. I found that the first shot was more difficult for After Effects to track than I had anticipated, so I’m going to mark that as another one to revisit down the line. The locked-off shot looks gorgeous, though. I only wish it lasted more than a second or two so I could appreciate the spinning stars while the observation drones pretend to be amused by anecdotes about the Dali Lama.

I’ve also been having a lot of trouble with the Premiere file I’d been using, so there’s an odd editing issue with the first Observation Dome window shot, and there are no crossfades or subtitles in this batch. I’ll probably just recreate the project file within the next couple of updates. Hopefully it’s just the Premiere file that’s gone bad, and the program itself isn’t reacting to something in the source videos.

Next up is another establishing shot of the station (which I’ll probably swap out for continuity reasons I’ll go into when I post it), then the Battle of the Line flashback, which will be preceded by what I’m sure will be an epic dissertation on just what happened during the Battle of the Line.

 

Model Credits:

Babylon 5: Ed Giddings
“Epsilon 3”: Nick Stevens
Epsilon Nebula: Amras
Starfury: Mark Kane
Starfury Wingart: Chris Guinn
Shuttle: Alexander Shareef

 

Babylon 5 2×01 Effects Update Supplemental- It’s Your Standard Warcruiser

It’s been a while since I updated this project, but a few weeks ago, it came back up in conversation. I realized at the time that I was now spending a lot of time sitting by a high-power desktop that wasn’t usually doing anything, and decided to take advantage of it.

I did end up trying Autodesk MatchMover. It handled the shot of Ivanova and Corwin from the last update with aplomb, but choked on the closing shot of the episode, which I expected to be the sharp, pointy crown jewel of difficulty on this project, containing both a lengthy camera move and a focal-length change.

I used the spare computer to re-render the second shot of the Agamemnon from the episode’s teaser, removing the odd tweening error I’d hoped no one would notice but which someone did. I then re-rendered it again when I got frustrated with the jump point opening effect. Yuri Parovin’s “A Call to Arms” jump point used a pre-made animated image map to accomplish the sparking that occurs before the point opens. Unfortunately, it was set for 30 frames per second, and I’m rendering the shots for this episode at 24 FPS. Lightwave appears to make an odd choice when it comes to splitting the difference in this case, keeping the timing correct by holding on the nearest frame, rather than making the transition within frames. The upshot was that the texture would hold for two frames, then skip a frame, then proceed normally for a time, giving the effect a noticeable stutter.

I asked around at the office, and figured out a plan of attack to recreate the effect procedurally. In Lightwave, it’s using “Previous Layer” as the input to a gradient shader, and in Maya, it’s called a Ramp Shader. Whatever it’s named, the way it works is to take a source image and redefine what its colors output as. In this case, grey becomes white, and both black and white become black.

I’d seen this technique described before, in a presentation by Bungie Studios (“Blowing S#!t Up the Bungie Way”) describing how they were able to get everything from flaming explosions to electrical arcing from one or two grayscale textures, and recognized it as how that jump point texture had to have been done, but I hadn’t realized until speaking with bottomless font of wisdom Chris Brown (my coworker, not the one you’re thinking of) how to translate that technique into the programs I use.

I tried a purely procedural solution, but it didn’t quite look right. The “energy ring” was too even, and it didn’t crinkle up at the center of the cone object the way the image map did. I eventually created a map in Photoshop to use as a baseline, and animated the “Previous layer” gradient instead. I threw in a semi-transparent layer of fractal turbulence in Lightwave which was gradually animated just so the energy effect wouldn’t repeat precisely every time. This was also the most contact I’ve had with Lightwave’s node-based surfacing, and I finally used some of the features that set it apart from the traditional surfacing options. The ability to directly feed one texture into multiple channels was a revelation, compared to the old way of having to copy-and-paste texture layers from color to specularity to glow to transparency and so on.

The upshot is, the new jump point now looks smooth opening up even at 24 FPS. Here’s the updated version of the shot.

I took another pass at the lighting rig for Rhys Salcombe‘s Minbari Warcruiser, brightening it considerably and switching the lights between the bottom ribs to single points rather than pairs of spots. I experimented with doing the effect with a luminance map, but it didn’t quite achieve the goal, so I stuck with doing it with lights. I also created a beaten-up version of the ship to represent the Trigati, to give some visual credence to the idea that the ship had been square in the middle of the biggest battle of the Earth-Minbari War when it deserted and hadn’t been able to make port for repairs in the twelve years since, and to differentiate it from the second Warcruiser that shows up at the end. It’s the first time I’ve done modeled damage on a spaceship, so that was fun.

The top fin has a blast mark on both sides, inspired by one of the shots of the Battle of the Line in “In the Beginning,” where a Starfury crashes into that section of a Minbari ship and blows out through the other side. I chewed away the lower left fin, and added a fairly large hole on the right side of the main hull, as if the right forward gun on the ship had misfired. I put a layer of procedural dirt over the whole ship, as well. I’m sure some people will think I’m overdoing it, given how much the Minbari outmatched Earth during the war, but there were a fair few shots of damaged Minbari ships in ITB, and I figured a Warrior Caste ship would be more likely than most to wade into the middle of the fight where they might take a few blows.

 

 

Babylon 5 2×01 Effects Update Part 003- A Miscommunication About the Time

And here comes the really ambitious part. I’ve decided to fill in an appropriately spinning starscape into the Observation Dome window. The first scene in the C’n’C was next up in the episode. One of the shots was locked off and at an angle, so there were no reflections, characters, or preexisting felt-and-rindstone star-fields to worry about. The other had all of those, as well as a relatively sweeping camera move that I had to match in Lightwave, rather than just rendering the view outside from the angle where you could see the most out the window and match-moving it in After Effects. In fact, it’s probably tied for the most difficult such shot in the episode. In light of that, I just temped it in here, superimposing the exterior render on top of the episode footage. I’ll come back later after I’ve done more of the simpler ones and gotten a better handle on how things work. Not to mention that I didn’t do the best job eyeballing the match move in Lightwave, and missed rendering one frame. Whoops.

Actually, I just remembered that my student copy of Maya is good for a while longer, and it included the unimaginatively named MatchMover. So, that should at least solve my “shots with big camera moves” problem. That only leaves the “thundering pain in the butt rotoscoping” problem. Why couldn’t the Earthforce uniforms have been, like, powder blue instead of navy? Why couldn’t Ivanova be blonde?

Anyway, here’s the scene in question, in glorious 720p. Even though that slightly uprezzes my 3D stuff, and drastically uprezzes the footage from the DVDs, it’s the only thing that I can think of that’ll stop YouTube from compressing it to death when I upload something at anything less. There’s also a photoshopped mockup for what the difficult shot might look like once it’s properly masked, and a modeler screenshot of the low-detail C’n’C I built so I could match the shots to exactly the right position and angle on B5.

I built it by matching camera angles to DVD screencaps in layout then shifting points in modeler until everything lined up. It’s a bit weird, probably since I had to guess about the original camera FoVs, but as long as the window was close, it was good enough. I’ll come back to it later and try to adjust the whole thing to be in proportion, especially the right side. The captain’s station is so screwed up, I don’t even know how it’s all supposed to fit together in the same space as the left side.

I was also really surprised when I figured out that the left wall wasn’t perpendicular to anything, not to mention how much bigger the room is in scale to the station than the corresponding part of the show model was.

Babylon 5: The Complete Second Season is available on DVD.

Babylon 5 2×01 Effects Update Part 002- Paying Off Karma

So, I’ve still been fiddling with this in the background, conducting some research and prep-stuff. Just to make myself happy, I found an on-line calculator for centrifugal artificial gravity and got a spin-rate for B5 that I could use with authority (once every 90 seconds). I added more of a self-lighting rig on Rhys Salcombe’s Minbari Warcruiser based on Sheridan’s ship in “The Lost Tales,” with lights in the recesses between the ribs, and one of B5 Scrolls’ interviews mentioned how the high deffy Epsilon 3 map was made for TLT (a recolored Mars with a bunch of gouges added from topographical surveys of the Grand Canyon so there’s somewhere for the Great Machine to go) so I shamelessly ripped off that tactic because I don’t trust my own painting ability and it looked fine in the movie.

I noticed something while splitting out the rest of the shots for this episode which amused me. The last establishing shot before the rogue Minbari ship arrives is the sun going down behind B5 and the station lights coming on. The first one after the Minbari are taken care of is the sun coming up. So, I have three choices, near as I can tell.

1- I can ignore it, like they did in the episode. 2- I can keep the establishing shots and redo all the scenes with the Minbari ships to be in local night, which will look cool and/or be an enormous pain in the butt, or 3- I can flip the establishing shots so it’s morning when Lennier explains Season One to Sheridan and Ivanova and evening when everyone meets up at the officer’s club and Sheridan gives his speech to an empty C’n’C. I’m leaning towards the last one as making the most logical sense.

Yesterday I cranked out the next shot in the episode, an establisher of Babylon 5. When I was looking at it, I realized that, as nice and proportional as Ed Giddings’ B5 is (the benefit of being the first one released after plan drawings of the original were released, IIRC), it’s really ’90s in terms of detailing. Since I’d want to replace it as soon as I could with a HD-level version anyway, I decided to phone it in a little and not do a proper shot breakout for this one yet, and just use it as a placeholder for now. Since I didn’t go through all that work, it meant I wasn’t really that upset when I looked at the finished product and found that I’d forgotten to reset my fake nebula radiosity from 30 fps to 24 fps, so the fill light is flickering, and that the red light in the center docking bay has shadow casting on, so the bay flickers off then back on when the little shuttle flies through it.

Babylon 5: Ed Giddings
“Epsilon 3”: Starbase1
Epsilon Nebula: Amras
Starfury: Mark Kane
Starfury Wingart: Chris Guinn
Freighter: Dave Hribarm
Shuttle: Alexander Shareef

Babylon 5 2×01 Effects Update Part 001- Yell. We’ll be there.

So, something I’ve talked about more than once is the pie-in-the-sky idea of an HD transfer of Babylon 5 which, by necessity, would require all-new visual effects, not unlike what was recently done with the original Star Trek series. A couple weeks ago, after school died down, I decided to relax with some nice, simple spaceship layouts, and begin attacking an episode of B5, possibly eventually doing the entire thing. For the moment, though, I’ve just done two visual effects shots in the teaser.

The episode I picked was the season two premiere, and the sequence is the very beginning, the introduction of Captain Sheridan aboard his soon-to-be-former command, the Agamemnon. I never seriously considered trying to be exceedingly faithful to the original shots, mostly because that’d be no fun. I mocked up a couple of images to show what such a recreation might look like, and, who knows, if someone was paying me I’d certainly be open to trying to match the originals as close as possible. But since it’s just me, I figure, why not indulge myself?

I’m not sure if I’m going to move forward on this anytime soon (I certainly haven’t ever completed a long-term solo production yet), but I prefer to live in hope. If I do, I’ll likely revisit earlier shots as I go along to adjust them.

Without further adieu, here’s the shot.

Omega Destroyer by Rhys Salcombe
Freighter by Dave Hribarm
Jumppoint by Yuri A. Parovin
Planet and Rings by Björn Jónsson

As a bonus, here’s a 1080p still from the sequence.