UWP

Mixing HoloLens 2D and 3D XAML Views in HolographicSpace

Matt Wilhelm

• 3 min read
Mixing HoloLens 2D and 3D XAML Views in HolographicSpace

If I can be a little honest, I feel like the HoloLens might be a solution in search of a problem.  But that's probably because it is so new and we are just beginning to understand its impact.  I think credit goes to Microsoft for understanding this, and that is likely why the device is not marketed to consumers.  But what about businesses?  Who can benefit from this technology?

The short answer is just about everyone; but one of the first value propositions for customers will be to add Holographic 3D views to already existing UWP applications.  So how do we do that?


If you browse the Microsoft Holographic Dev Center you'll see that the recommended way for mixing 2D and 3D space is to start with a Unity application and add your 2D XAML controls.  While this is compelling for some,  budgets have a hard time justifying a re-write for a platform that is still in its infancy.  There has to be another way.

DirectX

Enter DirectX, Microsoft's graphics technology that runs the HoloLens and other Windows devices.  The HoloLens runs on DirectX 11 specifically and supports high-performance 2D and 3D graphics.  Understanding the full programming model basically takes an academic education in computer graphics, but it essentially runs on the bare metal and offers really great performance.  But Microsoft has done its best to make this accessible, at least for the mature platforms.  In fact, there are several excellent articles on integrating Direct3D with XAML for different windows platforms but none for the HoloLens.

Looking through the link we see a series of XAML controls that can be dropped into an app to provide a DirectX canvas very easily, but these don’t translate to the holographic world.  Instead we have to use a HolographicSpace and a HolographicCamera to worry about.  Today that basically means we can't mix 2D experiences and 3D experiences together yet, but hey - it's a dev kit - right? :-)

So what's next?

We can build our apps in Unity and sprinkle in some XAML, or we can build our apps in XAML and sprinkle in some Holograms.  While this second options is clearly more complex, I think it will be the preferred method to add a little Holographic spice to our existing business logic. Let's give it a shot!

My first starting point was to download and install the HoloLens Emulator which comes with a shiny new Holographic DirectX 11 App Template.  If you create a new project with this template it gives you an example application of a spinning 3d cube and demonstrates how to construct a 3D application using the HolographicSpace and HolographicCamera using standard DirectX coding practices.

That is way beyond our scope.

All we want is to invoke this view from inside a XAML application.

On to the Code!

To accomplish our goal, we will:

  1. Copy the DirectX example code into a basic XAML application
  2. Override the threading model to be multi-threaded apartment
  3. Add a button on the main XAML page to invoke the DirectX view

So now, we are going to create a new "Blank App" UWP project and copy all of the code from the Holographic App Template into our new app.  There are a few things to watch out for:

  1. Copy all of the C# files over as well as everything in the Common & Content directories.
  2. Open the .csproj file and change the types for the HLSL files to match the .csproj from the Holographic App Template
  3. In the .csproj add the line to <Import Project="ms.fxcompile.targets" /> and make sure that file is also copied into the directory.
  4. In the .csproj file add DISABLE_XAML_GENERATED_MAIN; to the DefineConstants in order to hook the app startup.
  5. Now open the project in VS and build.

If you copied and referenced everything properly, you should essentially have the Holographic App Template code copied into a Blank App Template.  Because we added DISABLE_XAML_GENERATED_MAIN; Program.cs now hosts the main entry point of the application, which is currently set to bypass the XAML entirely via CoreApplication.Run().  Notice that the threading model has now also changed to [MTAThread] instead of [STAThread].  Now change the body of Program.Main() to:

global::Windows.UI.Xaml.Application.Start((p) => new App());

Next add a button to MainPage.xaml and add a click handler to MainPage.xaml.cs.  In that click handler we will use the ApplicationViewSwitcher API's to create a new 3D DirectX view and open it in a new window.  These are the same API's you would use on Desktop but since we are on a HoloLens and working with the HolographicSpace, the operating system will interpret this request and enter the "Volumetric" 3D view.

So what's happening here?

We are instantiating our AppViewSource just like we did in Program.cs before.  But now, we are using the CoreApplication.CreateNewView(IFrameworkViewSource viewSource) overload to create a new CoreApplicationView for our app.  On that view's thread we activate it and then tell the ApplicationViewSwitcher to show that view.

And that's it.

We've just successfully created a Hybrid XAML 2D and 3D Application.  Clearly we can expect changes in this space as Mircosoft evolves the API's and programming models available to us - but this should get you up and running and testing your apps today.