A short recap of what I’ve learned about Babylon JS over the last four weeks.

When I set out on this project a month ago my main goal was to better understand how to solve problems with Babylon JS. While I’m far from being an expert in this type of development, I’m happy with what I’ve learned. Babylon JS is a powerful new tool that I can start to use to solve problems for my customers and myself.

Initially, I was interested in making WebXR projects, but now that I’ve seen what Babylon JS can do, I’m open to a much wider range of projects and ideas.

I’m not going to go into detail about everything that I learned, but I’ll highlight a few things here.

3D Modeling via JavaScript

I spent the summer of 2018 learning how create 3D models in Blender. While I really enjoyed this type of work, some RSI issues with my hands prevented me from moving forward. Blender (along with most GUI based creation apps) was just too intensive on my hands because of the constant need to use a pointing device.

During the first week of this project, I learned the basics of mesh creation in Babylon JS. I was quickly impressed by the wide scope of features available for 3D modeling, and I recognized many of those features from my time in Blender. Having these capabilities available, while only needing to type a few lines of code, is incredibly valuable to me. I’ve even started to think about how I could use these APIs to build a 3D Modeling interface (in VR?) that I can use comfortably without the constant need of a mouse.

Animations

I used some animations during the Friday Projects for Day 5 and Day 10. Animations have always been hard for me to wrap my head around. I’ve done some basic animation work in Unity and Blender, but again, the GUI tools were hard for me to use and even hard for me to understand. Babylon JS has a simple system for animations that can be combined with JavaScript Events to create just about any type of timeline or sequence, all with just a bit of text in a text file.

Assets

Working with Assets using the Assets Manager is just plain awesome. It supports several content types from 3D models, textures, audio, and event text-based files for data. So far, I’ve only used this during the initial scene loading, but I’m excited about using it to request additional assets while a scene is running. Importantly, this has taught me about JavaScript promises in a way that I can visualize and understand.

GUI and Working with Data

Babylon JS has some notable features for rendering text and data onto a texture attached to a mesh. These features are what attracted me to Babylon JS in the first place. Many of the WebXR scenes that I want to build involve substantial amounts of data that I want to visualize and work with in a spatial setting. While the GUI features are not perfect, and they can be finicky at times, I’m happy to have access to these tools.

WebXR Features

Babylon JS makes it easy to get started with WebXR (VR and AR) development.

• Controllers: There is an abstracted control scheme that works across a number device types. I can also implement device-specific controls if needed
• Hand tracking for Oculus Quest!
• The pointer event system is simple for users while being powerful and flexible for developers. This system works with VR controllers, hand tracking, and even traditional input devices.
• Teleportation is easy to use, although a bit harder to customize.
• Mode changing: The entire process for entering and exiting VR mode is taken care of.

Moving Forward

Now that I’ve spent some time getting to know Babylon JS and what it has to offer, I can start to shift my attention away from learning and towards the projects that I want to build. That’s not to say what I don’t have a lot to learn, but to point out that I’m comfortable enough to move forward, learning as I go. Some areas that I want to improve include:

• WebXR controller input and interaction: I still have a lot to figure out such as how to use controller buttons that are not already mapped in the basic scheme, and how to interact with meshes with controllers and hands.
• Lighting: Most of my scenes have been using default lighting. I want to learn much more about taking control of lighting, as this can have an enormous impact on a project or scene.
• Scene management: How can I load content from multiple scenes? Can I move from one scene to another?
• Structuring code: How should I structure code in complex projects? How can I make API calls to load data while scene is already running?

Stats

• Education & Blogging Time: 16 hours
• Development Time (Friday Projects): 26 hours, 30 minutes
• Scenes & playgrounds: 8
• Demos published: 2
  • Babylon JS Day 5: Breathe
  • Babylon JS Day 10: Terrarium
• Blog posts: 21

Today is the final day of my A Month of Babylon JS education project and the last of the “Project Fridays”. I set out to build a simple golf demo for WebXR. I didn’t complete a full demo but I did get a few things done and I learned a lot about working with physics in Babylon JS.

I already had a golf club attached to one of the VR controllers, so I moved on to working on the ball. I created a prototype ball and positioned it where I could see it.

    var = globalBall = BABYLON.Mesh.CreateSphere("sphere1", 16, 2, scene);
    globalBall.position = new BABYLON.Vector3(-1, 0.2, 3);
    globalBall.scaling = new BABYLON.Vector3(0.05, 0.05, 0.05);

Then it was time to add some basic physics to the ball. I read though much of the documentation and found a couple of playgrounds to reference. I added physics at the scene level by defining gravity and selecting one of the available plugins.

    var scene = new BABYLON.Scene(engine);
    const gravityVector = new BABYLON.Vector3(0, -9.8, 0);
    const physicsPlugin = new BABYLON.CannonJSPlugin();
    scene.enablePhysics(gravityVector, physicsPlugin);

Then I activated physics on the ball by setting up a PhysicsImpostor. This seems similar to a rigid body in Unity. There are several shapes available and lots of options to customize.

 globalBall.physicsImpostor = new BABYLON.PhysicsImpostor(
      globalBall,
      BABYLON.PhysicsImpostor.SphereImpostor,
      { mass: 0.1, friction: 10, restitution: 0.5 },
      scene
    );

Just to have some fun, I decided to clone the ball in a loop and scatter the clones around the scene.

var y = 0;
    for (var index = 0; index < 20; index++) {
      var sphere = globalBall.clone("ball" + y);
      sphere.position = new BABYLON.Vector3(Math.random() * 20 - 10, 0.1, Math.random() * 10 - 5 + 5);
      y += 2;
    }

At this point I had a scene full of balls that could collide with each other, but I still couldn’t hit them with the club. My first attempt was adding a physics to the club mesh, but no matter what I tried I could not get it working. I also tried adding a simple box to the bottom of the club and adding physics to the box. That also failed and caused some weird issues with the position of the club on the controller. (I have no idea why a child object would affect the position of a parent object like this…)

My really bad workaround for today was to add physics to the motion controller itself, then modify the size and shape of the collider. You can see the dimensions of the collider in the imposterSize object below.

    // enable physics for the XR controllers
    const xrPhysics = xr.baseExperience.featuresManager.enableFeature(
      BABYLON.WebXRFeatureName.PHYSICS_CONTROLLERS,
      "latest",
      {
        xrInput: xr.input,
        physicsProperties: {
          restitution: 1,
          friction: 100,
          useControllerMesh: false,
          impostorSize: {
            height: 0.1,
            width: 0.05,
            depth: 1.6
          },
          impostorType: BABYLON.PhysicsImpostor.BoxImpostor
        },
        enableHeadsetImpostor: false
      }
    );

At this point I could finally move the balls with the club (actually, with the physics body on the controller… the club is just for show).

It may not look like much, but it took most of the day. While I can certainly move the balls with the club now, it is far from something that feels natural, or even just fun. I think in the future I’ll remove the physics from the controller and find another way to apply force to the ball from the club. I think I can do something with mesh intersection and the velocity of the club, but that is for another day.

Next week I’ll write a recap post reviewing what I’ve learned this month. I’m far from being an expert at Babylon JS, but I’m getting comfortable working with it and I’m happy to add it to the set of tools with which I build applications and sites.

Today I worked on importing a golf club model and attaching it to a VR controller as a child object. I started off by setting up an AssetsManager for the scene, then importing a model. I’m using the blue club from the Minigolf Kit from Kenney game assets. I’m really starting to like the AssetsManager feature. It seems like I can count on anything loaded this way to be ready and available to other parts of the scene by the time the scene is done loading.

    // Import assets
    var assetsManager = new BABYLON.AssetsManager(scene);
    const path = "../assets/models/";
    assetsManager.addMeshTask("mesh task", "", path, "club_blue.glb").onSuccess = function (task) {
      globalClub = task.loadedMeshes[0];
      globalClub.position = new BABYLON.Vector3(0, 1, 2);
      globalClub.name = "club";
    };
    assetsManager.load();

After loading the mesh in the mesh task, I assign it to a global variable so I can get a reference to it later. I’m sure there is a better way to do this, but this works for now.

Attaching the golf club to the controller was a lot harder than I thought it would be. I started by referring to the playground that I mentioned yesterday. First, I needed to swap out the box mesh with the club that I imported, hence the global variable mentioned above. Then I used a reference to the motion controller called mesh as the parent object for the club. Getting the club positioned and rotated relative to the controller was the hard part. The rotate method was confusing me. It took me a while to realize that this method wanted radians and I was entering degrees…

The order of the rotation and translations was also important. When the club is attached to the parent mesh, I set its rotation to match the parent (controller), then rotate around the X axis, then rotate around the Y axis. Finally, move the club “down” the Y axis just a bit.

// WebXRDefaultExperience
    const xr = await scene.createDefaultXRExperienceAsync({
      floorMeshes: [ground]
    });

    xr.pointerSelection.detach();

    xr.input.onControllerAddedObservable.add((inputSource) => {
      inputSource.onMotionControllerInitObservable.add((motionController) => {
        motionController.onModelLoadedObservable.add((mc) => {
          let mesh = inputSource.grip;
          if (motionController.handedness[0] !== "l") {
            globalClub.setParent(mesh);
            globalClub.position = BABYLON.Vector3.ZeroReadOnly;
            globalClub.rotation = mesh.rotation;
            globalClub.rotate(BABYLON.Axis.X, 1.57079, BABYLON.Space.LOCAL);
            globalClub.rotate(BABYLON.Axis.Y, 3.14159, BABYLON.Space.LOCAL);
            globalClub.locallyTranslate(new BABYLON.Vector3(0, -0.1, 0));
          }
        });
      });
    });