Blog

A short recap of what I’ve learned about Babylon JS over the last four weeks.

When I set out on this project a month ago my main goal was to better understand how to solve problems with Babylon JS. While I’m far from being an expert in this type of development, I’m happy with what I’ve learned. Babylon JS is a powerful new tool that I can start to use to solve problems for my customers and myself.

Initially, I was interested in making WebXR projects, but now that I’ve seen what Babylon JS can do, I’m open to a much wider range of projects and ideas.

I’m not going to go into detail about everything that I learned, but I’ll highlight a few things here.

3D Modeling via JavaScript

I spent the summer of 2018 learning how create 3D models in Blender. While I really enjoyed this type of work, some RSI issues with my hands prevented me from moving forward. Blender (along with most GUI based creation apps) was just too intensive on my hands because of the constant need to use a pointing device.

During the first week of this project, I learned the basics of mesh creation in Babylon JS. I was quickly impressed by the wide scope of features available for 3D modeling, and I recognized many of those features from my time in Blender. Having these capabilities available, while only needing to type a few lines of code, is incredibly valuable to me. I’ve even started to think about how I could use these APIs to build a 3D Modeling interface (in VR?) that I can use comfortably without the constant need of a mouse.

Animations

I used some animations during the Friday Projects for Day 5 and Day 10. Animations have always been hard for me to wrap my head around. I’ve done some basic animation work in Unity and Blender, but again, the GUI tools were hard for me to use and even hard for me to understand. Babylon JS has a simple system for animations that can be combined with JavaScript Events to create just about any type of timeline or sequence, all with just a bit of text in a text file.

Assets

Working with Assets using the Assets Manager is just plain awesome. It supports several content types from 3D models, textures, audio, and event text-based files for data. So far, I’ve only used this during the initial scene loading, but I’m excited about using it to request additional assets while a scene is running. Importantly, this has taught me about JavaScript promises in a way that I can visualize and understand.

GUI and Working with Data

Babylon JS has some notable features for rendering text and data onto a texture attached to a mesh. These features are what attracted me to Babylon JS in the first place. Many of the WebXR scenes that I want to build involve substantial amounts of data that I want to visualize and work with in a spatial setting. While the GUI features are not perfect, and they can be finicky at times, I’m happy to have access to these tools.

WebXR Features

Babylon JS makes it easy to get started with WebXR (VR and AR) development.

• Controllers: There is an abstracted control scheme that works across a number device types. I can also implement device-specific controls if needed
• Hand tracking for Oculus Quest!
• The pointer event system is simple for users while being powerful and flexible for developers. This system works with VR controllers, hand tracking, and even traditional input devices.
• Teleportation is easy to use, although a bit harder to customize.
• Mode changing: The entire process for entering and exiting VR mode is taken care of.

Moving Forward

Now that I’ve spent some time getting to know Babylon JS and what it has to offer, I can start to shift my attention away from learning and towards the projects that I want to build. That’s not to say what I don’t have a lot to learn, but to point out that I’m comfortable enough to move forward, learning as I go. Some areas that I want to improve include:

• WebXR controller input and interaction: I still have a lot to figure out such as how to use controller buttons that are not already mapped in the basic scheme, and how to interact with meshes with controllers and hands.
• Lighting: Most of my scenes have been using default lighting. I want to learn much more about taking control of lighting, as this can have an enormous impact on a project or scene.
• Scene management: How can I load content from multiple scenes? Can I move from one scene to another?
• Structuring code: How should I structure code in complex projects? How can I make API calls to load data while scene is already running?

Stats

• Education & Blogging Time: 16 hours
• Development Time (Friday Projects): 26 hours, 30 minutes
• Scenes & playgrounds: 8
• Demos published: 2
  • Babylon JS Day 5: Breathe
  • Babylon JS Day 10: Terrarium
• Blog posts: 21

Today is the final day of my A Month of Babylon JS education project and the last of the “Project Fridays”. I set out to build a simple golf demo for WebXR. I didn’t complete a full demo but I did get a few things done and I learned a lot about working with physics in Babylon JS.

I already had a golf club attached to one of the VR controllers, so I moved on to working on the ball. I created a prototype ball and positioned it where I could see it.

    var = globalBall = BABYLON.Mesh.CreateSphere("sphere1", 16, 2, scene);
    globalBall.position = new BABYLON.Vector3(-1, 0.2, 3);
    globalBall.scaling = new BABYLON.Vector3(0.05, 0.05, 0.05);

Then it was time to add some basic physics to the ball. I read though much of the documentation and found a couple of playgrounds to reference. I added physics at the scene level by defining gravity and selecting one of the available plugins.

    var scene = new BABYLON.Scene(engine);
    const gravityVector = new BABYLON.Vector3(0, -9.8, 0);
    const physicsPlugin = new BABYLON.CannonJSPlugin();
    scene.enablePhysics(gravityVector, physicsPlugin);

Then I activated physics on the ball by setting up a PhysicsImpostor. This seems similar to a rigid body in Unity. There are several shapes available and lots of options to customize.

 globalBall.physicsImpostor = new BABYLON.PhysicsImpostor(
      globalBall,
      BABYLON.PhysicsImpostor.SphereImpostor,
      { mass: 0.1, friction: 10, restitution: 0.5 },
      scene
    );

Just to have some fun, I decided to clone the ball in a loop and scatter the clones around the scene.

var y = 0;
    for (var index = 0; index < 20; index++) {
      var sphere = globalBall.clone("ball" + y);
      sphere.position = new BABYLON.Vector3(Math.random() * 20 - 10, 0.1, Math.random() * 10 - 5 + 5);
      y += 2;
    }

At this point I had a scene full of balls that could collide with each other, but I still couldn’t hit them with the club. My first attempt was adding a physics to the club mesh, but no matter what I tried I could not get it working. I also tried adding a simple box to the bottom of the club and adding physics to the box. That also failed and caused some weird issues with the position of the club on the controller. (I have no idea why a child object would affect the position of a parent object like this…)

My really bad workaround for today was to add physics to the motion controller itself, then modify the size and shape of the collider. You can see the dimensions of the collider in the imposterSize object below.

    // enable physics for the XR controllers
    const xrPhysics = xr.baseExperience.featuresManager.enableFeature(
      BABYLON.WebXRFeatureName.PHYSICS_CONTROLLERS,
      "latest",
      {
        xrInput: xr.input,
        physicsProperties: {
          restitution: 1,
          friction: 100,
          useControllerMesh: false,
          impostorSize: {
            height: 0.1,
            width: 0.05,
            depth: 1.6
          },
          impostorType: BABYLON.PhysicsImpostor.BoxImpostor
        },
        enableHeadsetImpostor: false
      }
    );

At this point I could finally move the balls with the club (actually, with the physics body on the controller… the club is just for show).

It may not look like much, but it took most of the day. While I can certainly move the balls with the club now, it is far from something that feels natural, or even just fun. I think in the future I’ll remove the physics from the controller and find another way to apply force to the ball from the club. I think I can do something with mesh intersection and the velocity of the club, but that is for another day.

Next week I’ll write a recap post reviewing what I’ve learned this month. I’m far from being an expert at Babylon JS, but I’m getting comfortable working with it and I’m happy to add it to the set of tools with which I build applications and sites.

Today I worked on importing a golf club model and attaching it to a VR controller as a child object. I started off by setting up an AssetsManager for the scene, then importing a model. I’m using the blue club from the Minigolf Kit from Kenney game assets. I’m really starting to like the AssetsManager feature. It seems like I can count on anything loaded this way to be ready and available to other parts of the scene by the time the scene is done loading.

    // Import assets
    var assetsManager = new BABYLON.AssetsManager(scene);
    const path = "../assets/models/";
    assetsManager.addMeshTask("mesh task", "", path, "club_blue.glb").onSuccess = function (task) {
      globalClub = task.loadedMeshes[0];
      globalClub.position = new BABYLON.Vector3(0, 1, 2);
      globalClub.name = "club";
    };
    assetsManager.load();

After loading the mesh in the mesh task, I assign it to a global variable so I can get a reference to it later. I’m sure there is a better way to do this, but this works for now.

Attaching the golf club to the controller was a lot harder than I thought it would be. I started by referring to the playground that I mentioned yesterday. First, I needed to swap out the box mesh with the club that I imported, hence the global variable mentioned above. Then I used a reference to the motion controller called mesh as the parent object for the club. Getting the club positioned and rotated relative to the controller was the hard part. The rotate method was confusing me. It took me a while to realize that this method wanted radians and I was entering degrees…

The order of the rotation and translations was also important. When the club is attached to the parent mesh, I set its rotation to match the parent (controller), then rotate around the X axis, then rotate around the Y axis. Finally, move the club “down” the Y axis just a bit.

// WebXRDefaultExperience
    const xr = await scene.createDefaultXRExperienceAsync({
      floorMeshes: [ground]
    });

    xr.pointerSelection.detach();

    xr.input.onControllerAddedObservable.add((inputSource) => {
      inputSource.onMotionControllerInitObservable.add((motionController) => {
        motionController.onModelLoadedObservable.add((mc) => {
          let mesh = inputSource.grip;
          if (motionController.handedness[0] !== "l") {
            globalClub.setParent(mesh);
            globalClub.position = BABYLON.Vector3.ZeroReadOnly;
            globalClub.rotation = mesh.rotation;
            globalClub.rotate(BABYLON.Axis.X, 1.57079, BABYLON.Space.LOCAL);
            globalClub.rotate(BABYLON.Axis.Y, 3.14159, BABYLON.Space.LOCAL);
            globalClub.locallyTranslate(new BABYLON.Vector3(0, -0.1, 0));
          }
        });
      });
    });

I didn’t have much time to work on Babylon JS today, but I did have an idea for what I want to build for the project on Friday. Since receiving the Oculus Quest 2 last fall I’ve been playing Pro Putt quite a bit with friends. Sometimes we talk about different game mechanics that we would build if we were making a golf game. I’m not going to try to build a full golf game, at least not any time soon. Instead, I’m going to see if I can get a usable demo done this week. I’ll layout what I hope build here.

• Make or import a simple golf club mesh. Attach it to the as a child object to the controller so that it moves when the VR controller moves. Hide the Oculus controller model.
• Make a simple ball. Easy. Place a ball at a fixed location in a scene. Bonus: press a button on the controller to respawn the ball after I hit it.
• Learn how to use physics in Babylon JS. Tap the ball with the end of the club to make the ball move. This will take some iteration to find something that feels natural.
• Make some sort of target. A hole and flag to aim for or just a large collider.
• If there is any time after finishing the items above, I’ll try to make a few interesting golf holes with some terrain.

I got started today by learning how to attach a mesh to the VR controllers. This playground has an example of placing small box meshes to the inside of the VR controllers. I think I can modify this to attach a club mesh. I’ll tackle that problem tomorrow.

Today I learned about teleportation in WebXR with Babylon JS. I used the default experience helper to add WebXR to my scene, so teleportation is available by default. The only thing I needed to provide is an array of meshes that the user can teleport on. In this case I created two group planes and passed them to the floorMeshes option. There are also a couple of methods to add and remove floor meshes to the teleportation system.

    var groundMat = new BABYLON.StandardMaterial(scene);
    groundMat.diffuseColor = new BABYLON.Color3.FromHexString("#9ba8b8");

    var ground1 = BABYLON.Mesh.CreateGround("ground1", 20, 20, 2, scene);
    ground1.material = groundMat;
    var ground2 = BABYLON.Mesh.CreateGround("ground2", 20, 20, 2, scene);
    ground2.position = new BABYLON.Vector3(0, 0, 25);
    ground2.material = groundMat;


    // WebXRDefaultExperience
    const xrDefault = await scene.createDefaultXRExperienceAsync({
      floorMeshes: [ground1, ground2]
    });

There are two kinds of teleportation.

• Direct teleportation will draw a straight line and can be used when the target mesh is in sight
• Indirect teleportation will draw a parabolic path and is useful when the target mesh is not in sight. The most common use case for this is when teleporting to platform that is above the player position. This type of teleportation can be deactivated if needed. It also has a few properties that can be customized.

The “landing zone” is a set of meshes the make up the object that indicates where the user will teleport. This is something that I want to customize, as the default appearance and size of this don’t appeal to me. By default, this landing zone object is about ten times larger than I think it should be… The documentation has some information on how to customize the appearance and behavior of the landing zone.

One of the core features that Babylon JS includes with the teleporter is the ability to turn or change directions during a teleportation action. I’m not a fan of this and I try to turn it off whenever a game lets me do so. Lucky for me, they include a property that I can toggle off to remove this behavior.

 xrDefault.teleportation.rotationEnabled = false;

For the most part, I really like the teleportation features that Babylon JS includes. They packed a ton of options and customization into this feature. I need to spend some time customizing the landing zone, but aside from that I’ll use most of the default options.

This is the last full week of the A Month of Babylon JS project. Most of the projects I want to build with Babylon JS are related to VR, so this week I’m going to learn everything I can about the WebXR features that Babylon JS has to offer.

I started today with the Diving Deeper documentation on WebXR. This page has some information about setting up a scene with WebXR, as well as some comparisons of WebVR vs WebXR.

The most basic setup involves adding this line to a scene

const xr = scene.createDefaultXRExperienceAsync();

As I mentioned last week, this helper is asynchronous, so the default template is not suitable. Here is a basic template that I use to make a new XR scenes.

<!DOCTYPE html>
<html lang="en">

<head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    <title>Babylon JS WebXR Template - 2021.03.22</title>
    <style>
        html,
        body {
            overflow: hidden;
            width: 100%;
            height: 100%;
            margin: 0;
            padding: 0;
        }

        #renderCanvas {
            width: 100%;
            height: 100%;
            touch-action: none;
        }
    </style>
    <script src="https://cdn.babylonjs.com/babylon.js"></script>
    <script src="https://preview.babylonjs.com/gui/babylon.gui.min.js"></script>
    <script src="https://cdn.babylonjs.com/loaders/babylonjs.loaders.min.js"></script>
    <script src="https://code.jquery.com/pep/0.4.3/pep.js"></script>
    <script src=https://preview.babylonjs.com/inspector/babylon.inspector.bundle.js></script>
</head>

<body>
    <canvas id="renderCanvas" touch-action="none"></canvas> <!-- touch-action="none" for best results from PEP -->

    <script>
        const canvas = document.getElementById("renderCanvas"); // Get the canvas element
        const engine = new BABYLON.Engine(canvas, true); // Generate the BABYLON 3D engine


        window.addEventListener("DOMContentLoaded", async function() {
            // get the canvas DOM element
            var canvas = document.getElementById("renderCanvas");
            // load the 3D engine
            var engine = new BABYLON.Engine(canvas, true);
            // createScene function that creates and return the scene
            var createScene = async function() {
                // create a basic BJS Scene object
                var scene = new BABYLON.Scene(engine);
                scene.debugLayer.show();

                const camera = new BABYLON.ArcRotateCamera("camera", -Math.PI / 2, Math.PI / 2.5, 3,
                    new BABYLON.Vector3(0, 0, 0));
                camera.upperBetaLimit = Math.PI / 2.2;
                camera.lowerRadiusLimit = 2;
                camera.upperRadiusLimit = 50;
                camera.setPosition(new BABYLON.Vector3(0, 1.5, -2));
                camera.setTarget(new BABYLON.Vector3(0, 1.5, 0));
                camera.attachControl(canvas, true);

                // create a basic light, aiming 0,1,0 - meaning, to the sky
                var light = new BABYLON.HemisphericLight("light1", new BABYLON.Vector3(0, 1, 0),
                    scene);

                // Our built-in 'ground' shape. Params: name, width, depth, subdivs, scene
                var ground = BABYLON.Mesh.CreateGround("ground1", 20, 20, 2, scene);
                var material = new BABYLON.StandardMaterial(scene);
                material.alpha = 1;
                material.diffuseColor = new BABYLON.Color3.FromHexString("#9ba8b8");
                ground.material = material;


                // WebXRDefaultExperience
                const xrDefault = await scene.createDefaultXRExperienceAsync({
                    floorMeshes: [ground]
                });
                const xrHelper = xrDefault.baseExperience;


                // return the created scene
                return scene;
            };

            // call the createScene function
            var scene = await createScene();

            // run the render loop
            engine.runRenderLoop(function() {
                scene.render();
            });

            // the canvas/window resize event handler
            window.addEventListener("resize", function() {
                engine.resize();
            });
        });
    </script>
</body>

</html>

The template above uses the default experience helper, which includes several built-in features such as pointers to interact with UI, teleportation, and the basic process of entering and exiting XR modes. There is also a basic experience helper available, which has fewer features by default but may be useful when more control is needed. For now, I’m going to use the default version.

Last week I was focused on interacting with the 3D cards. I was using the pointer event system that was added when I setup the default experience helper. Those features allowed me to interact with GUI controls that were rendered using the AdvancedDymaincTexture. Toward the end of the day on Friday I added some code that allowed me to grab and move the cards around in 3D space. This was also using the pointer system. While moving objects with the pointers is an interesting (and powerful) feature, I also want to be able to use the controllers/hands to interact with objects. Throughout the week I’ll work on this and some other ideas.

Today is Project Friday so I took the day to work on the scene that I’ve been building throughout Week Three. Rather than produce a demo scene like in the previous weeks, today I just made incremental progress the interactive 3D GUI cards for VR.

I started by adding some images to the item cards. All the cards use the same image for now, but eventually the cards will load images related to their post data. I added a new GUI Image object to the panel on the item card.

// Load an image from the project assets folder 
var image = new BABYLON.GUI.Image("image", "../assets/images/babylon-month/2021.03.02-A-Month-of-Babylon-JS.png");
  image.height = "400px";
  panel.addControl(image);

Then I went to work on the detail card that I mocked up yesterday. The detail card is a box object that has a plane object as a child. The plane contains an AdvancedDynamicTexture where I draw the data that needs to be displayed.

I added a ScrollViewer to the card instead of using a StackPanel. I’m not sure if I’ll keep this around or not. It’s nice to know that I can make scrollable content in VR, but I’m not convinced this is the best way to handle substantial amounts of content. I added a TextBlock and setup the padding and sizing to work with the ScrollViewer. I referenced this playground when I was building the card.

const detailCard = BABYLON.MeshBuilder.CreateBox("detailCard", { height: 2.2, width: 3, depth: 0.2 });
    detailCard.position = new BABYLON.Vector3(1.8, 1.6, -0.3);
    detailCard.rotation = new BABYLON.Vector3(0, 0.5, 0);
    detailCard.scaling = new BABYLON.Vector3(0.3, 0.3, 0.3);
    makeGrabbable(detailCard);

    const detailPlane = BABYLON.MeshBuilder.CreatePlane("plane", { height: 2, width: 3 });
    detailPlane.parent = detailCard;
    detailPlane.position.z = -0.11;
    detailPlane.position.y = 0.1;
    var advancedTexture = BABYLON.GUI.AdvancedDynamicTexture.CreateForMesh(detailPlane);
    advancedTexture.background = "white";

    var sv = new BABYLON.GUI.ScrollViewer();
    sv.thickness = 8;
    sv.color = "gray";
    sv.width = 1;
    sv.height = 1;
    sv.background = "white";

    advancedTexture.addControl(sv);

    var detailNote = new BABYLON.GUI.TextBlock("detailNote");
    detailNote.fontFamily = "Tahoma, sans-serif";
    detailNote.text = "This is a test";
    detailNote.textWrapping = true;
    detailNote.color = "black";
    detailNote.fontSize = 24;
    detailNote.resizeToFit = true;
    detailNote.paddingTop = "5%";
    detailNote.paddingLeft = "30px";
    detailNote.paddingRight = "20px";
    detailNote.paddingBottom = "5%";
    detailNote.horizontalAlignment = BABYLON.GUI.Control.HORIZONTAL_ALIGNMENT_LEFT;
    detailNote.verticalAlignment = BABYLON.GUI.Control.VERTICAL_ALIGNMENT_TOP;
    detailNote.textHorizontalAlignment = BABYLON.GUI.Control.HORIZONTAL_ALIGNMENT_LEFT;
    detailNote.textVerticalAlignment = BABYLON.GUI.Control.VERTICAL_ALIGNMENT_TOP;

    detailNoteTextBlock = detailNote;

    sv.addControl(detailNote);

Then it was just a matter of updating the button action on the info cards to send data to the detail card. I cheated here. I couldn’t find a way to query the TextBlock. It has something to do with the fact that it’s rendered insider of the AdvancedDynamicTexture. For now, I just made a global variable that holds a reference to the detailNoteTextBlock. While not ideal, there is only one of these cards in the scene so this will work until I can find a better way.

 button1.onPointerUpObservable.add(function () {
    let detailCard = scene.getMeshByName("detailCard");
    if (!detailCard.isEnabled()) {
      detailCard.setEnabled(true);
      detailNoteTextBlock.text = decodeHtml(item.contentEncoded);
    } else {
      detailCard.setEnabled(false);
      detailNoteTextBlock.text = "";
    }
  });

Here is a preview of selecting an item card and sending data to the detail card. The sample data I’m working with contains HTML markup, but the TextBlock doesn’t know what to do with it. I need to create some better sample data…

I rewrote the process that positions the item cards. When I load the sample data, I get back an array of 15-25 items. I decided to chunk this array in an array of sub-arrays, then loop over both arrays to set positions for each card. I also created an anchor object so I can move all the cards together.

function createCards(scene, items) {
  var anchor = new BABYLON.TransformNode("itemCards");
  anchor.position = new BABYLON.Vector3(0, 2.2, 0);

  let chunked = chunk(items, 5);
  let j = 0;
  for (section of chunked) {
    j++;
    let i = 0;
    for (item of section) {
      i++;
      let x = 0.25 * i;
      let y = -0.35 * j;
      let positionObject = {
        x: x,
        y: y,
        z: 0
      };
      createCard(scene, item, positionObject, anchor);
    }
  }
}

Then I moved on to VR interaction. There are a ton of VR features that I want to add over time. I started with a simple “grab” feature. While looking for examples on the Babylon JS forum I found this post and adapted the sample code to suit my needs. The card meshes are grabbable, but the GUI content (and the plane) are not. To get around that I added an extra section to the bottom of the cards. This grab feature relies on the pointer event system. A user points a controller at a card, presses the trigger button to start the grab, then moves the card around the scene. Press the trigger again to drop the card. Not a bad start.

That’s all that I had time for today. This weekend I may spend a bit more time on this. I’d like to start working with some better sample data. I have a personal database of my VR games and apps. I might use that instead of the blog post data. While this demo scene is just part of the learning process for me this month, the features I’m working on are something that I want to use on several projects in the future.

Today I learned how to add VR support to my scene. Until now I’ve been using a template that Babylon JS provided in their Getting Started guide. I had to replace most of that code with an async version to be able to use the WebXR features that Babylon JS offers.

// Add scene code here
const createScene = () => {
  // Scene, camera, lights
  const scene = new BABYLON.Scene(engine);
  scene.debugLayer.show();
  scene.clearColor = BABYLON.Color3.FromHexString("#35013f");

  const ambientLight = new BABYLON.HemisphericLight("light", new BABYLON.Vector3(0, 10, 0));
  ambientLight.position = new BABYLON.Vector3(-1, 20, 30);
  const directionalLight = new BABYLON.DirectionalLight("DirectionalLight", new BABYLON.Vector3(0, -1, -3), scene);
  directionalLight.position = new BABYLON.Vector3(-1, 20, 30);

  const camera = new BABYLON.ArcRotateCamera("camera", -Math.PI / 2, Math.PI / 2.5, 3, new BABYLON.Vector3(0, 0, 0));
  camera.upperBetaLimit = Math.PI / 2.2;
  camera.lowerRadiusLimit = 1;
  camera.upperRadiusLimit = 50;
  camera.setPosition(new BABYLON.Vector3(0, 0, -20));
  camera.attachControl(canvas, true);

  // Our built-in 'ground' shape. Params: name, width, depth, subdivs, scene
  var ground = BABYLON.Mesh.CreateGround("ground1", 20, 20, 2, scene);
  var groundMaterial = new BABYLON.StandardMaterial(scene);
  groundMaterial.diffuseColor = new BABYLON.Color3.FromHexString("#9ba8b8");
  ground.material = groundMaterial;

  return scene;
};

// Call the scene
const scene = createScene(); //Call the createScene function

//Register a render loop to repeatedly render the scene
engine.runRenderLoop(function () {
  scene.render();
});
// Watch for browser/canvas resize events
window.addEventListener("resize", function () {
  engine.resize();
});

window.addEventListener("DOMContentLoaded", async function () {
  // get the canvas DOM element
  var canvas = document.getElementById("renderCanvas");
  // load the 3D engine
  var engine = new BABYLON.Engine(canvas, true);
  // createScene function that creates and return the scene
  var createScene = async function () {
    // create a basic BJS Scene object
    var scene = new BABYLON.Scene(engine);
    scene.debugLayer.show();
    // create a FreeCamera, and set its position to (x:0, y:5, z:-10)
    var camera = new BABYLON.FreeCamera("camera1", new BABYLON.Vector3(0, 1.5, -5), scene);

    // target the camera to scene origin
    camera.setTarget(new BABYLON.Vector3(2, 1.5, 0));
    // attach the camera to the canvas
    camera.attachControl(canvas, false);
    // create a basic light, aiming 0,1,0 - meaning, to the sky
    var light = new BABYLON.HemisphericLight("light1", new BABYLON.Vector3(0, 1, 0), scene);

    // Our built-in 'ground' shape. Params: name, width, depth, subdivs, scene
    var ground = BABYLON.Mesh.CreateGround("ground1", 20, 20, 2, scene);
    var material = new BABYLON.StandardMaterial(scene);
    material.alpha = 1;
    material.diffuseColor = new BABYLON.Color3.FromHexString("#9ba8b8");
    // material.diffuseColor = new BABYLON.Color3(1.0, 0.2, 0.7);
    ground.material = material;



    // Add XR support
    // Enable XR
    var experience = await scene.createDefaultXRExperienceAsync({
      // define the floor meshes
      floorMeshes: [ground]
    });

    // return the created scene
    return scene;
  };

  // call the createScene function
  var scene = await createScene();

  // run the render loop
  engine.runRenderLoop(function () {
    scene.render();
  });

  // the canvas/window resize event handler
  window.addEventListener("resize", function () {
    engine.resize();
  });
});

I’m using the createDefaultXRExperienceAsync helper. Over time I need to learn how to customize the VR features a bit, but for now I’m going to use all their default settings.

Once I got the scene loaded in VR on my Oculus Quest 2, the first obvious change that I needed to make was scaling the cards down in size. When I built the card earlier this week, I sized them with “full unit” values to keep things simple. While that looks OK on the desktop browser, in VR these cards were huge. It was just a matter of scaling the entire card (with all child objects) at the end of my createCard function.

card.scaling = new BABYLON.Vector3(0.15, 0.15, 0.15);

Then I turned my attention to the event for the buttons on the cards. When a user presses this button, I want to show an expanded detail UI on another object in the scene. The scene will only have one “detail card” but may contain any number of the smaller “info cards”. Think of this as a master-detail interface, but in 3D space. First, I added a placeholder object for the detail card. I made this much larger than the item cards and positioned it behind them for now. Assigning a name in the constructor is important. I’ll need that to query this item in the scene graph.

const detailCard = BABYLON.MeshBuilder.CreateBox("detailCard", { height: 4, width: 6, depth: 0.4 });
    detailCard.position = new BABYLON.Vector3(5, 4, 6);

Then I updated the button event. I used a method on the scene called getMeshByName() to get a reference to the detail card in the scene, then toggle the isEnabled property on the detail card. Later, this button will also send data related to the info card to the detail card for display. While the info card contains a few preview items, the detail card will contain full text, images, etc.

button1.onPointerUpObservable.add(function () {
    let detailCard = scene.getMeshByName("detailCard");
    detailCard.setEnabled(detailCard.isEnabled() ? false : true);
    console.log("card button was clicked");
  });

Tomorrow I’ll get to work connecting all these features together. I want the user to be able to grab and move the info cards. I’d like to replace the button the card with a controller interaction. For example, say the user picks up a card and presses the A button to show the detail card. I also want to the detail card to show up in a position relative to the users’ position, not a fixed spot like it is now. I’m not sure how much I’ll be able to get done during the time I’ve allotted for the Friday Project, but I’m sure I’ll have something interesting to write about when I’m done.

Here are a couple of rough previews of the info and detail cards, recorded in the Oculus Quest 2.