IKEA Walkthrough v2.3.1

Wed, Mar 3, 2004

Reiner Fink (old buddy and probably responsible for some of the more interesting comments in the NT source) sent me a link to the IKEA Walkthrough:

IKEA is a fully immersive, 3D environmental adventure that allows you to role-play the character of someone who gives a shit about home furnishings. In traversing IKEA, you will experience a meticulously detailed alternate reality filled with garish colors, clear-lacquered birch veneer, and a host of NON-PLAYER CHARACTERS (NPCs) with the glazed looks of the recently anesthetized.

Skewed view

Tue, Feb 24, 2004

David Cumps is playing around with XAML and skewing UI controls.  He found a bug which causes some rendering artifacts when doing a skew of a listbox.  I tried some of his XAML out on the latest and greatest bits and didn't find the bug.  So, this is either a video driver issue, or it is something that we've fixed since the PDC.  Here is a (badly encoded -- thanks mspaint.exe) gif of a skewed list box:

A skewed list box

Back from the dead

Sat, Feb 21, 2004

I don't know how many people care, but I'm sorry that I haven't been posting lately!  I'll try to make that up some.  Anyway, here is an update of what has been happening:

  • Went to Cabo San Lucas for a week and relaxed.  It was great to sit in the sun at the beach, order a bucket of beers and eat tacos.
  • Adopted two dogs.  Their names are Lady and Cocoa.  Lady is one and a half years old and was rescued from a meth lab and was probably abused.  She hasn't shown many signs of it, but you never know what is going to trigger memories of abuse.  She had a litter of puppies when she was really young.  Cocoa is the one puppy for which the rescuers didn't find a home.  He was her favorite puppy, so we adopted them as a pair.  More details and pictures to come later.
  • I celebrated the one year anniversary of my blog by not posting.  You can find the first post here from Janurary 9, 2003.
  • Read lots and lots of books.  I've been reading ebooks on my PocketPC on the bus to and from work every day.  I love ebooks!
  • Started writing some ASP.Net code for a picture gallery over at bedafamily.com.  I'd love to finish this up so that I can easily post lots of photos.

I've started recovering from the hibernation that is winter in Seattle.  I want to start talking more about what is going on in Avalon and life in general.

Home Theater update

Mon, Dec 22, 2003

I haven't been posting in a while as I've been working on my new Home Theater!

Here is the final setup:

  • Sanyo Z2 LCD projector.  1280x720 widescreen.
  • Paradigm Monitor speakers.  I got Monitor 7s in front with the CC-370 center and the ADP-370 dipole surrounds.  I picked these up in Canada on Chris' recommendation.
  • SVS 20-39PCi subwoofer.  This thing is a beast!  It is over 3 feet high and can really shake the walls.
  • Denon AVR-3803 reciever.  There was a great deal going on these.  Otherwise it would have been way out of my price range.
  • ReplayTV 5504 as a TV tuner.  I got one of these as it was priced wrong at Circuit City.  Normally it is ~$450 with 3 years of service, but they were selling them for $150 with 3 years of service as they had updated the pricing for no included service but the units they still had did have the free service.
  • XBox.  Some games (like Amped 2) do 720p natively.  They look pretty sweet.

I also built my own ceiling mount for the projector and had a friend help me build and paint a screen.  I'll put some pictures up when things get further along.

Changeables desconstructed

Mon, Dec 22, 2003

Frank Hileman (no blog, Frank?) has been pounding us (the Avalon team) with some great questions.  Most of these have been conversations in comments, but I figured that I should bring this to the front page, so to speak.

Of of Frank's concerns, at least as expressed in my comments section, is that we are essentially over engineering with the Changeable pattern.  Specifically, he says this:

The non-intrusive part of Changeable has a const flag on stock brushes, etc. This means the user must clone these and write to a copy. This is simple and easy to understand.

The intrusive part of Changeable is the part which introduces the idea of "used" and changes the simple concept of "object reference" into something complicated. From what you have explained, the intrusive part of Changeable exists only to allow simultaneous multi-threaded writers to the scene graph.

I think that the point around threading is a red-herring here. 

Changeable Requirements

The main problem that changeables are trying to solve is that we want to have, under most circumstances, these changeable API objects (Brush, Pen, etc.) act like value types and other times we want them to act like refernce types.  We essentially want to walk a fine line and get advantages from each.  Frank, forgive me if you've heard a bunch of this before, but I think that this is useful to put together in one place.

With respect to value types, we want the simple user semantic to be: Modify my local copy and have it be disconnected or "severed" when I pass it in to another object.  When rendering this means that I can do this:

SolidColorBrush scb = new SolidColorBrush();
scb.Color = Colors.Blue;
myDrawingContext.DrawGeometry(scb, null /*pen*/, myGeometry1);
scb.Color = Colors.Red;
myDrawingContext.DrawGeometry(scb, null /*pen*/, myGeometry2);

and have my two geometries be drawn with different colors.  Since many times the drawing context is writing in to a metafile like data structure, if we used pure reference semantics both geometries would be drawn as Red. 

Another example is setting properties in to a set of elements:

SolidColor Brush scb = new SolidColorBrush();
scb.Color = Colors.Blue;
myElement1.Background = scb;
myElement2.Background = scb;

// in some other code...
((SolidColorBrush)myElement1.Background).Color = Colors.Red;

I think, in most cases, if some user walks up to an element, such as myElement1 above, and sets the background brush, he doesn't expect that this will have the side effect of changing the background brush of another element.  Side effects like this would make for an unstable system.

Value types don't solve all of our problems though!  We also want the efficiencies and advanatages of making these be reference types.  Specifically, we want to have to have hierarchies, not have to copy the entire thing around and be able to save space by having multiple places reference the same thing.  We imagine a world with lots of elements getting their values via styling.  Because of this, we want the normal case be that a user puts a value in to a style block somewhere and then that object (via reference) gets widely shared, perhaps between hundreds of elements.  We obviously want to make the per-reference cost as small as possible here.  Going with a pure value type system won't work.

The Builder Pattern

The first cut at solving this problem was called the Builder Pattern.  This pattern was a generalization of the StringBuilder pattern.  Specifically, for each type, there was a companion, or Builder type.  The main type was immutable.  In other words, any write operations would raise an exception.  So, for SolidColorBrush, we wrote a SolidColorBrushBuilder.  SolidColorBrush was immutable while SolidColorBrushBuilder was a fully writeable object.  All of the APIs in the system (property types on element, parameters in to DrawingContext methods, etc.) took the non-builder types.  To create one of those non-builder types, the user had to do one of the following:

  • Create the target type directly by using a constructor.
  • Create a builder, configure it, and call the ToXXX method on it to construct the main type.
  • Create a builder by initializing it off of its companion type.  Make a delta and call the ToXXX method to construct the main type.

From one point of view, this is a great system.  It is very explicit (at compile time) what is editable and what isn't.  However, some early users of our system (ad-hoc usability) ended up really disliking this system.  The need to constantly deal with both types was onourous.  It also meant that the user could never just index directly in to a type and change it.  To affect any change, it was always necessary to create the builder, make the change, and set the old value in.

The Changeable Pattern

In response to a hail of internal criticism on the Builder Pattern we went back and came up with the Changeable pattern.  The idea is that we would have one type object with an immutable bit.  Once the immutable bit gets set, all further changes to that object will throw.  This opens the way up to runtime errors (instead of compile time errors) but makes certain "delta" scenarios much easier to deal with.

However, we still wanted to solve the problems listed in the first part of this post.  Specifically, we didn't want to leave it up to the user to manually flip the immutable bit.  In this way we defined certain types of operations that would automatically create a copy and flip the immutable bit for you if necessary.  We then went further and added advanced modes so that those APIs either make a copy but don't flip the immutable bit or the APIs just take a reference.  These last modes (whereby something like an element holds on to a mutable brush) were never possible with the Builder pattern.

We also added a notification mechanism so that someone who does hold on to an object that isn't immutable can learn when that object changes.

So, do you guys (Frank?) like the more explicit builder pattern better?  More questions?

DX interop in Avalon

Mon, Dec 22, 2003

Vince (who I knew in high school as Steve) wrote in my comments:

How can I get to the underlying DX9-esque interface I know you guys gotta be using this stuff. I don't want to deal with EllipseGeometry objects and the like, gimme DrawPrimitive, push buffers, and pain-in-the ass ITexture objects.

Don't worry dude!  The details are still being worked out, but we know that we have to make it possible for people to interop with old code, and with DX.  At the very least, you will be able to host another hwnd window that is written using GDI/User or DX.  The details on when and how DX will get accelerated, and how it will play with the rest of the system are still TBD.  At the very least, when you go full screen with DX you will own the device.

For simple 3D stuff (load up a model and spin it around like a 3D icon) we will have our own 3D API that doesn't pretent to offer all of the flexibility of D3D.

What specific scenarios are you interested in?  My thinking is that you might want to write a tool and have the UI done with Avalon and the "canvas" of the tool done via D3D.  In this scenario, you are in a windowed situation and Avalon is on top.  The 3D stuff is running in a window in an Avalon app.  Another scenario that might be interesting is that you have a D3D app/game and you are using Avalon to drive 2D type UI inside of the game environment.

I can't promise that all of the scenarios will be doable, but let me know what is most important to you.

Holiday parties

Mon, Dec 22, 2003

I went to a great holiday party last night thrown by my neighbors, Bob and Carol.  We moved in quite a while ago but hadn't really had a chance to interact with any of our neighbors yet.  This was a great opportunity for us to not only chat with our hosts, but also others from around the neighborhood.  It turns out that the internet makes the world even smaller.  Bob had run across my blog and already knew more about me (or at least my work!) than I knew about him.

I also chatted with our neighbors on the other side, Gary and Marge.  Gary is a long time photographer that just recently went digital with a Canon 10D and a sweet lens setup.  He sent me a link to some of his images.  These were done with film -- a Pentax 6x7 that he recently sold.  I really need to get off of my butt and put together a gallery of some of my better up on the net.

Avalon Changeable pattern

Fri, Dec 5, 2003

Greg Schechter (whose office is right next to mine) has a new blog and has posted some great info on the Avalon Changeable pattern.  This pattern was originally developed for the graphics space but is useful in many other places.  I was going to write about changeables at some point but Greg beat me to it.

Avalon Job Opening

Thu, Dec 4, 2003 Apply here for a job working on Avalon.  Specifically, this job is for our very cool timing and animation team.  We are a pretty dynamic team so the actual responsibilities may include other parts of Avalon graphics depending on who we get and what their skills are.

Avalon Visuals

Tue, Dec 2, 2003

Drew Marsh posted an interesting article entitled "Avalon Dissected."  It is great to see people taking an interest in Avalon at this level.  However, he got some of the details around Visuals wrong.

The graphics system in Avalon is new and is built to take advantage of the hardware.  Beyond this, there is a software engine for the cases when the hardware isn't supported.  On top of this, we've built a "scene graph" system that natively supports animation and remoting.  In the normal case, the actual rendering of the scene graph is done on a dedicated composition thread where third party code isn't allowed to run.  The description of everything on screen and some primitive animation information is passed to this thread so that the rendering can happen independently of what is going on with the regular UI thread.  In the case when you are running an Avalon application from a remote machine (via Terminal Services or Remote Desktop) with an advanced client, this composition thread will run on the remote machine and the actual rendering of the scene graph will happen without any network traffic.  Some animations will be able to run on the remote machine without network traffic also. 

From the programming model point of view, a node in our scene graph is called Visual.  There are different types of visuals that are intended to be used for different purposes.  It is also important to note that UIElement derives from Visual so every UIElement is-a Visual.  (At one point an Element had-a visual but this was changed with the rearchitecture that ChrisAn talks about.)

Every Visual implements an interface called IVisual.  There is no situation where a third user should be implementing IVisual.  The reason that we created this interface is so that we can use explicit interface implementation to hide the methods on IVisual.  We didn't want to pollute the public object model for UIElement (and Button, etc.) with things that most users weren't going to be using.

A Visual has the following capabilities:

  • Children with a transform on each child
  • Opacity
  • Clip
  • Content (defined differently for each visual type)
  • Hit testing services
  • Coordinate transformation services
  • Bounding box services
  • And in the future: blend mode and effect

For each visual, the content of that visual is rendered followed by the children, from left to right.  The Z-order of any children are completely implied by their order in the visual tree.

Different specializations of Visual provide for specific types of content and for different flavors of public programming model.  Here are the main ones:

  • ContainerVisual.  This Visual has all of its capabilities reflected publically and has no intrinsic content.
  • DrawingVisual.  This Visual also has all of its capabilties reflected publically.  Its content is 2D vector data that is stored with it.  You can think of it holding on to a "metafile."  The way you change the content is call DrawingVisual.RenderOpen, get a DrawingContext and draw with that.
  • RetainedVisual.  This guy is special.  He is meant to be derived from.  This is the Visual that UIElement derives from.  All of the services and properties are available either via private functions (RenderOpen) or via the IVisual interface only.  This visual can work in two modes:
    • Fully retained.  This is the default case.  In this case the visual is exacly like a DrawingVisual except the IVisual members are not reexposed in a public way.  RenderOpen is also protected instead of private.
    • On demand retained.  If the derived class implements IRetainedRender and sets the protected RenderBounds property, then IRetainedRender.Render will be called back as necessary to fill in the "metafile" for this Visual.  They system looks at the tree and decides, based on the RenderBounds, if this Visual is on screen.  If it is, then the system calls IRetainedRender.Render and the derived class writes into the "metafile" via a DrawingContext passed.  The system then caches that metafile.  If the Visual is then pushed off screen for some reason(scrolling, perhaps), then the system is free to throw away the cached metafile to save space.  The user can force the system to throw away the metafile by calling the protected RetainedVisual.InvalidateVisual call.

When it comes time to render (scheduled via the Dispatcher class and coordinated with the animation system and layout) the system will walk the tree and decide what is and isn't on screen.  It will then call render on any RetainedVisuals as necessary and cache the result.  It also updates the data on the other thread for what is on screen so that the render can happen async from what is going on on the UI thread.  In this way, there is no "render" happening on the UI thread in the normal case.  Instead we are "compiling" the scene graph down to a simpler representation to run asynchronously on another thread or on another machine.

The bits that we shipped in the PDC don't have this system fully implemented.  For example, the composition thread isn't running completely async and we don't have rich TS support in.  Come beta, more of our system will be completed and up and working.

I'm sure that this is confusing and that I've missed some stuff, so feel free to ask questions and I'll do my best to answer.