juliklive

Outputting overscan Action scenes with 3-D cameras

When you do set extensions, and especially when you use tracked cameras, it is often necessary to do 2-d transformations outside of your original frame. The most common thing to do is zoom out on your original footage, revealing your set extension. For this, you basically need two images, with one image being bigger and with both of them perfectly aligned, pixel to pixel. Of course you can manually scale the bigger image to match, but instead you can use a little of Flame math to do it precisely.

Just adjusting the size of your setup does not work as you would expect to - it does not increase the field of view of your camera, but instead changes the output resolution. When you want to output a background that will align with our original picture and yet contain overscan you need to know how much overscan to apply in terms of the camera FOV.

Here is how.

Outrageous

Wanted to buy myself a copy of Alias Sketchbook Pro (won't be calling it Autodesk). From Autodesk itself (Sketchbook Pro is one of the items they retail directly online), this will cost me (without the CD delivered, mind you!)

At the same time, on Amazon US

This brings us to an all-time record fucking high of 3.21 times price difference between Europe and US. If we concider the tax, it would be 2.7 times difference. Autodesk should be proud - they are now more fucked up than Adobe in terms of ripping us.

Needless to say, a response from the product manager at Autodesk did not contain a single word of meaningful information on why this fucked up pricing scheme is even possible. And needless to say, no way in hell I am buying from people with this attitude, at least not something for my personal use.

Some serious EU legislation for this crap is in order.

Say something...

Architecture behind tracksperanto-web

So it's been quite a while since tracksperanto-web went production, with more than 300 files converted so far with great results. I hope some sleepless nights for matchmovers worldwide have already been preserved and nothing could be better.

I would like to bring a little breakdown on how Tracksperanto on the web is put together (interesting case for background processing). The problem with this kind of applications (converters to be specific) is that the operation to convert something might take quite some time. To mitigate that, I decided to write the web UI for Tracksperanto in such a way that it would not be blocking for processing, so extensive use of multitasking is made.

On my servers I am still using FCGI since it works and frees me from any additional process management (to reload apps it's enough to restart the webserve, no monit, no god - no nothin). However, when you want to do background-processing in Ruby you need to make choices - and fork() was the best choice for this case. When I start the app, the following happens:

Lighttpd starts up and spins off the FCGI dispatcher, supported by rackup. To make Rack and Sinatra function properly I had to rewrite the lighttpd fix middleware like this.
The rackup script, in turn, forks() a dispatcher process. The beauty of this is that the dispatcher process will die when lighttpd will tell the FCGI process to quit.
Then the work is split. The main FCGI process presents the default Sinatra app (supported by Rack::Cache). When a new upload is done, the file is saved by the Sinatra process and a new record in the database is made.
The dispatcher process constantly polls the database for new jobs, and if one is found within a transaction it will be fetched, marked in the DB as being busy - we need a concurrent database for this - and then the processing itself happens.
The dispatcher process spins up a new forked child just to process the job, again via fork(). Tracksperanto jobs use alot of processor time and alot of memory, and with Ruby nobody can guarantee that jobs won't leak memory. fork() is highly beneficial in this way since after the job has been completed the forked worker will just die off, releasing any memory that has been consumed in the process. When one day I will switch to REE the gain will be even bigger since the Tracksperanto code will be copy-on-write.
When processing is taking place, the worker process writes the status into memcached. Tracksperanto is designed so that every component can report it's own status and a simple progress bar can be constructed to display the current state. So basically we constantly (many times per second) write a status of the job into memcached - percent complete and the last status message. To let the user see how processing is going, I've made an action in Sinatra that quickly polls memcached for the status and returns it to the polling Javascript as a JSON hash.

This scheme has the following benefits:

Status reporting does not load the database (not needed and the information is hardly crucial).
Zero memory leakage
No Ruby daemon processes (the daemons gem is the worst piece of crap I've ever seen, overspinning processes and almost always failing either to restart or to stop the processes that have been spun up).
Start/stop control is tied into the webserver.

For jobs in the database I am using a hand-rolled solution resembling delayed_job.

Here's how the Rackup file looks:

require File.expand_path(File.dirname(__FILE__) + "/../app")
require File.expand_path(File.dirname(__FILE__) + "/../lighty_workaround")
require 'rack/cache'

worker_controller_pid = fork do
  log_path = TW_ROOT + "/log/worker.log"
  main_logger = Logger.new(log_path, "daily")
  ActiveRecord::Base.establish_connection(TW_CONFIG["db"])
  loop do
    # We might have lost the connection when we forked, so reconnect
    ActiveRecord::Base.verify_active_connections!
    job = Job.find_first_pending

    # Process the actual job in the child, 
    # so that any memory that gets overconsumed will be freed
    # upon exit independently of the Ruby GC. We can easily do this
    # since we don't need to receive any info back from the processor
    if job
      child_pid = fork do
        # Internally reopen the logger
        worker_logger = Logger.new(log_path)

        # Ensure all sockets are reinstated in the child process
        MessageBus.reconnect!
        ActiveRecord::Base.verify_active_connections!

        # Run the job
        job.exec!

        # Goodbye
        worker_logger.warn "Completed the job #{job.to_param}"

        # And exit explicitly and nicely
        exit!(0)
      end

      main_logger.warn("Spun up worker #{child_pid} for #{job.to_param}")

      # Spin up a thread that will wait for the child to terminate
      # to avoid <defunct> children
      Thread.new { Process.waitpid(child_pid) }
    end
    sleep(1) # Do not torture the DB all the time
  end
end

# Establish a thread that will wait for the worker manager process to quit
Thread.new { Process.waitpid(worker_controller_pid) }

builder = Rack::Builder.new do
  use Rack::ShowExceptions 
  use LightyWorkaround
  use Rack::Cache, :metastore => 'file:/tmp/rack/cache/meta',
        :entitystore => 'file:/tmp/rack/cache/body'
  map "/" do
    run TW.new # Run the Sinatra app
  end
end

run builder

A few important snags:

 Job.find_first_pending

actually does the transactional code that will also mark the job as being in progress if found (to prevent other dispatchers from hijacking it).

 Thread.new { Process.waitpid(child_pid) }

uses Ruby threads for what they do best - waiting for something to happen. In this case, we monitor every process that has been spun up and ensure that it gets killed properly. The same is done for the loop process controlled by the FCGI dispatcher.

    # Ensure all sockets are reinstated in the child process
    MessageBus.reconnect!

If you did any work with Ruby subprocesses you should know that most sockets (and thus Ruby objects holding them) will disconnect when you fork(). MessageBus is a simple wrapper around Memcached calls that we use and ActiveRecord has to reconnect as well.

   # Run the job
   job.exec!

The exec! method will actually capture and record any exceptions that might occur during processing. When such a thing happens, the Job class itself will see to it that the file which failed will be emailed to me. The same happens when a job completes succesfully, but then no attachment is sent. To send messages with attachments, I am using the pony gem. Also a special status flag is written to memcahed so that the user gets a concise error message.

To prevent abuse, I do some security-by-obscurity and the code for tracksperanto-web will not be published (but you can get a peek at it if you contact me privately).

Say something...

Announcing Tracksperanto, a 2D tracking converter

I've released Tracksperanto, a small app that can convert 2D tracking data between many apps. In my work I often have to do 2D tracking and then transition to camera solves - some apps do it better, some apps do it worse and you seldom know which solver will give you a better result. I've taken a look at Mincho Marinov's Survey Solver converter but found it inadequate (bloated code that is not easy to extend, only runs from within Maya and MEL is not the best language to process text). So I've written my own system to accomplish just that and released it as open source.

It works like this: you use a file with 2D tracks as a source. Currently, Tracksperanto can import Syntheyes tracks, PFTrack .2dt files, MatchMover track exports, Flame .stabilizer setups, Shake tracker exports, Nuke and Shake scripts. It scavenges all the trackers which are in the file, and then outputs converted versions for Syntheyes, MatchMover, Nuke, Shake and PFTrack with correct coordinates - so you can only track once and then solve everywhere. Boujou will accept a Shake text file as well.

Tracker correlation is also passed along where possible. You can also process your tracks to scale the whole comp together with all tracker keyframes, move the tracked points left and right and slip all the tracker keyframes in time (things which are VERY time-consuming when done manually, especially if your shot is extended at the beginning during that pesky reedit next Monday).

I've found this approach especially helpful for working with large sequences at high resolutions. Compositing apps usually don't need to load the whole image into memory to do tracking, while matchmoving apps always do - they are built around OpenGL and insist on loading full frame into the viewport, which quickly saturates your cache and makes tracking painfully slow. If you track in a compositing app, you have the benefit of very fast tracking (thanks to either Flame's expensive storage or to partial loading tricks employed by Nuke and Shake).

So feel free to try it out. Tracksperanto can be run locally (you need to have Ruby running on your computer, Macs come with Ruby preinstalled), or used online - just upload your file and get all the converted versions back packaged into a neat zip archive.

Tracksperanto can deal with large files (that's how it has been conceived) - I'm converting Flame stabilizer files with tens of trackers and thousands of keyframes, up to 5 megs in size, and dealing with Shake scripts with hundreds of Tracker nodes (Tracksperanto will actually evaluate your Shake script and recover all Stabilize, MatchMove and Tracker node information).

And of course, it's extensible and forkable, so TDs can use it to develop custom conversion workflows if needed.

Tracksperanto is part of of Guerlla DI, my set of Ruby utilities for post.

Say something...

We need Rubyforge more than ever

...especially in these times of Ruby version transition.

..I have to go hunting for updated repositories, audit the code lightly to check if they’re high on crack or not and generally waste everyones time.

GitHub folks are very fine, but I do hold them accountable for transforming the Ruby package management landscape into whale guts, Linux model or otherwise. This is not the case where truth (any other truth than "gforge is a piece of shit but nobody will invest the time to rewrite it") will emerge out of chaos.

The rubyforge gems model may not be perfect, but damnit people, when there’s a gem update I know that it has actually been tested somewhat and it’s not just whatever random point HEAD happens to be at, at that point in time, by some random Joe who just bought TextMate.

Well said.

Say something...

_path routes considered harmful

There's been some talks lately on relative versus absolute routes and where they should be used.

Personally, I consider the point moot. If you have such a powerful tool at your disposal (Rails) there is one thing you certainly should do, once and for all - make all URLs both absolute and canonical. Here is the reasoning for that:

Canonical URLs work in standalone documents (like PDF and downloaded pages). Anywhere you transition from displayable HTML to something else, you have to go canonical
Atom and RSS feeds are busted when you use relative URLs. Granted, there is the xml:base but some feed readers just don't give a shit (making all your links dead). Nothing is more frustrating than having unclickable links. NetNewsWire, last time I checked, did not honor xml:base. This situation obliges you to rewrite all URLs that go out to feeds
The same for sent email - there is no mechanism for specifying a URL base for hyperlinks in mail messages.

Also, if you promote web service consumption off your app (REST or otherwise) you oblige the consumer to do path-munging for every redirect, every cross-reference and every parse just to drill down your site structure (all these ugly url.path = something things come from that).

So if you ask me, I'd just hardwire *_path to raise or play through to *_url. These are processor cycles (and wire bytes) well spent, because they simplify the consumption of your website outside of anything except the browser, and free you from deciding every time which one you need. Additionally, modern times never cease to amaze us with regards to ways our content can bleed out of the browser.

At some point I had to write a plugin to enforce canoncal/absolute URLs in text segments, and for Inkpot (the engine running this site) I use canonical URLs for the absolute most of the links and cross-references. The exception is the stuff going through the Camping router, but my gut feeling tells me that the use of Camping will be abolished here anyway.

Just my 2 cents.

Say something...

Stop the red tape before it gets any further

The industry is in an interesting state now. As VFX get more adoption across all disciplines of filmmaking, there's more and more demand for our work ad at the same time things get devalued. There is much less experimentation (except at the top-notch facilities like ILM and DD), much less exploration - but the amount of work is actually increasing. Everybody seems to want more, faster and better than before, for less money. Good facilities are closing, not able to sustain their own weight in the economic downturn. However, there is one thing that still strikes me, after having worked in the industry for the past 3 years.

The red tape.

When I come to have a problem in the Ruby world, with some web technology or anything of that nature - it's usually a matter of hours to get to a canned answer on one or other blog, wiki or forum. Asking people helps, and there is a "show source" making things even more accessible. There are whole universes where raw knowledge in the form of code roams free, ready to be taken. There is a thriving community of people sharing ideas, giving keynotes and presentations, putting these online for free. Every motion spurs discussion and natural, community QA. And there are open source projects, where you can examine the solutions to problems you face daily in their entirety - to learn and assimilate has never been so easy. It's a thriving, lively ecosystem.

By comparison, the post/VFX business is a swamp. I'll make use of my lowly position to say it out loud. Blogging post people are few and far between. And even when they do post, there is this uncanny feeling of "saying without saying", the pressure that is mounted behind. As artists. we literally abstain from our own process, preserving the only right we might have left for ourselves - to put the work done on showreels for viewing to "audiences of close friends". I talked to a very talented Danish animator once, and he told me that it's usually around a year for a studio to give you even that - that meager playblast you have done for their show, contributing your sould, blood and sweat.

No, meetings are awful

Dare Obasanjo on iPhone:

Every once in a while I may be running late for a meeting and have to either send a mail out to the attendees telling them I'll be late or cancel the meeting. Neither of these options is available on the iPhone.

No, meetings themselves are awful. Exchange-scheduled meetings doubly so. It's the enterprise stigma.

Say something...

Finally integrated

Thanks to the advice by Jeremy Lightsmith I've got CC.rb running on the very same machine I've checked it out to. Works fine with both git and subversion now. While this might not have been true in the past, git integration indeed works perfectly (it's polling instead of notifiers, though - which are in fact no more than pimped post-commit hooks and a maintenance chore).

Say something...

Server move

Yet another change of hosting to evaluate some VPS options. As a consequence, this site will now load much slower for those on our side of the pond (Europe/Russia) and much faster in the States.

This as a result of either overpriced (Netherlands) or incompetent (Germany) european VPS offerings. BTW, the prices seem ever more absurd knowing that European bandwidth is much cheaper (but the crucial balance on the RAM front is just not there).

Say something...