Babylon JS Day 2

Today I worked through chapters three and four of the Babylon JS Getting Started guide. After working in Unity for a couple of years, most of these concepts are not new to me, just the way that Babylon JS implements them. Some things that stood out as interesting to me are

  • MeshBuilder has a method called ExtrudePolygon that can be used for complex extrusions. I wonder what other methods are available… While I can create some basic models in Blender, I have trouble using tools like that because they require the constant use of a pointing device. I will happily create models via code when I can.
  • Lots of information on how Babylon JS handles animations. Animations can be imported along with models and used in a scene, but I can also code some animations directly in Babylon JS. There are a handful of interesting playgrounds with sample code to check out.
  • There is a handy method called movePOV that will move an object using it’s forward vector. When using this I don’t have to calculate differences in vectors based on orientation of the object I’m moving. I can just pass a Vector3 that indicates the movement from the perspective of the object.
  • Chapter 4 was a brief introduction to collisions. It explained how to test for collision between two objects. I need to learn more about the default bounding boxes and if I can customize these shapes.

A Month of Babylon JS

Throughout the month of March, I’m going to learn everything I can about Babylon JS. Over the last year I’ve spent a bit of time with A-Frame, Three JS, and Babylon JS. I explained my reasoning for settling on Babylon JS in the most recent episodes of Project Update.

I have a history of spending too much time on courses and educational content, and not enough time building something with the tools I’m learning. This month I’m going to try to break that cycle by planning how I work on this project.

  1. I’m going to stop work an hour early each day (Monday through Thursday) to spend time working through the Babylon JS Getting Started guide and documentation.
  2. I’m going to take each Friday in March as a day to spend building small projects, with an emphasis on what I’ve learned over the previous days.

My hope is that this approach will place some limits on the project while giving me plenty of time to spend on it. While this is clearly in the “side project” column of work, I still consider this actual work and will treat it as such. I’ll be tracking my time, taking lots of notes, and reporting my progress during Project Update recordings with Dave.

One of my goals for this project is to learn to think in the way that Babylon JS and other similar JavaScript frameworks want me to think. These environments all approach problems in ways than I’m not accustomed to. I’m just not used to thinking in in the JS way yet. With a background in FileMaker development, lots of work in PHP, Swift, and C#–I’m used to solving problems in ways that don’t always make sense in the JS context.

I got started on this project yesterday, March 1, 2021, with the Babylon JS Getting Started Guide. I worked my way through chapters one and two. While most of this was just a refresher of things I already knew, I did pick up a few things.

  • I don’t always need to pass a reference to a scene when creating an object. When I leave this null the value will default to the current scene.
  • I can save/export Babylon JS scenes to a .babylon file format that can be viewed or imported elsewhere.
  • I can import entire scenes or portions of scenes from these saved files. This seems like an interesting way of making something like Prefabs in Unity.
  • Babylon JS has some interesting ways of making copies and instances of objects. I want to learn more about instances and what they inherit from their base objects.

Data Entry with SwiftUI and UIKit

Version 1.0 of Retrospective Timelines was written completely in SwiftUI and while I’m excited to keep working in SwiftUI, there were some issues and limitations that it imposed on my app.

You can’t really talk about SwiftUI without someone pointing out that it’s not “ready” to be used in production yet. I mostly disagree with that sentiment, but when it comes to data entry forms it rings true. Many of the common UI controls in SwiftUI have the most basic implementation possible, often with little to no ability to customize them.

SwiftUI TextField for example.

  • No way to dismiss the keyboard (you can press Return, but a lot of users don’t think to try that)
  • Text selection in the field is really flakey.
  • No SwiftUI equivalent of TextView, so multi-line text is not possible
  • Speech to Text fails. I’ve tried this on every device I have and it fails 100% of the time. It will insert the first character from the first word you speak, then quit listening. I reported this bug months ago.

These issues and limitations were reflecting poorly on the app and giving my users the wrong impression so I decided to do something about it.

My first attempt was to wrap the UIKit controls in SwiftUI wrappers, but I had little success. I cobbled together a sort-of-working version of UITextField, but it was buggy and unreliable. It was also super slow. Rather than keep spending time on it, I decided to just write the entire screens in UIKit.

In Retrospective Timelines all data entry is done in a modal view. There are only two data entry forms and a limited number of ways to open them. I had already disregarded SwiftUIs sheet presentation due to its limitations and instead was using an environmental variable to get the hosting View Controller to use for presentation. All had to do was change the View Controller that I was presenting from the SwiftUI hosting controller I was showing, to a new UIKt version.

I don’t know a ton about UIKit, so it took me a couple of days to make these forms. I used a storyboard for the view controllers and some XIBs for small elements like collection and table view cells. The forms are basically static UITableViewController objects with a ton of customization. Setting these up and accounting for all of the edge cases was a lot more work than the same thing in SwiftUI. I found myself missing the way SwiftUI handle the state of data.

The end result turned to better than I thought possible, so I can’t really complain. Not only did I work around the limitations of the SwiftUI controls, I also ended up with a far better version of the Color and Image pickers for Timelines.

Generating Shareable Text Images

One of the core features of Retrospective Timelines is the ability to share an Event as an image. I’m working in SwiftUI on this project so my first attempt was to see if I could render SwiftUI views as images. Last June Erica Sadun wrote a post about rendering SwiftUI on macOS Mojave that gave me some ideas to try. There is some sample code in that post that renders a UIView as an image, but the view is hosting SwiftUI views in the first place. I tried to adapt this to my needs in the app, but I quickly ran into issues.

First, I couldn’t find a way to make SwiftUI/iOS render view content that was not on screen. If I wanted to make a large resolution image I would need to scale it to fit on screen on an iOS device in order to save it as an image. Second, scaling the image in this manner introduced all sorts of scaling artifacts. The backgrounds look OK but the text was awful.

The next I thing I tried ended up being my “good enough” solution. I rendered a UIView offscreen with all of the text views manually positioned and added to this view.

Content View contains a simple UI. At the top is an Image view that is showing a live preview of the final image that we are rendering and sharing. There is a text field to enter a string of text and a simple color picker object in a scroll view. The function called `createViewToRender()` is doing most of the work here. I create some views, position them, and add them to a parent object.

struct ContentView: View {
    
    @Environment(\.viewController) private var viewControllerHolder: UIViewController?
    @State var selectedColor: String = "user_purple"
    @State var displayText: String = ""
    
    var body: some View {
        VStack(alignment: .center) {
            
            Image(uiImage: self.renderViewAsImage)
                .resizable()
                .mask(RoundedRectangle(cornerRadius: 10))
                .overlay(RoundedRectangle(cornerRadius: 10)
                    .stroke(Color.gray, lineWidth: 0.5)
            )
                .scaledToFit()
                .shadow(color: Color.gray, radius: 1, x: 0, y: 0)
                .padding()
            
            TextField("Enter a string", text: self.$displayText)
                .textFieldStyle(RoundedBorderTextFieldStyle())
                .padding()
            
            ScrollView(.horizontal) {
                ColorPicker(selectedColor: self.$selectedColor)
            }
            
            Spacer()
            sharingActionButton
                .padding()
            Spacer()
            
        }
        .frame(minWidth: nil, idealWidth: nil, maxWidth: 800, minHeight: nil, idealHeight: nil, maxHeight: nil, alignment: .center)
        .navigationBarTitle(Text("Share"))
    }
    
    var sharingActionButton: some View {
        Button(action: {
            self.shareEvent()
        }, label: {
            VStack {
                Image(systemName: "square.and.arrow.up")
                    .font(Font.body.weight(.bold))
                    .padding()
                Text("Share as image and/or text")
                    .font(.footnote)
            }
        })
    }
    
    private func roundedFont(fontSize: CGFloat, weight: UIFont.Weight) -> UIFont {
        if let descriptor = UIFont.systemFont(ofSize: fontSize, weight: weight).fontDescriptor.withDesign(.rounded) {
            return UIFont(descriptor: descriptor, size: fontSize)
        } else {
            return UIFont.systemFont(ofSize: fontSize, weight: .regular)
        }
    }
    
    private var renderViewAsImage: UIImage {
        createViewToRender().renderAsImage()
    }
    
    private func createViewToRender() -> UIView {
        
        let labelPositionX: CGFloat = 40
        let labelWidth: CGFloat = 944
        
        let eventNamePositionY: CGFloat = 140
        let eventHeight: CGFloat = 300
        let eventFontSize: CGFloat = 60
        
        let background = UIView()
        background.backgroundColor = UIColor(named: self.selectedColor)!
        background.frame = CGRect(x: 0, y: 0, width: 1024, height: 576)
        
        
        let text = UILabel()
        text.frame = CGRect(x: labelPositionX, y: eventNamePositionY, width: labelWidth, height: eventHeight)
        text.text = self.displayText
        text.numberOfLines = 3
        text.textColor = UIColor.white
        text.font = roundedFont(fontSize: eventFontSize, weight: .bold)
        text.textAlignment = .center
        
        background.addSubview(text)

        return background
        
    }
    
    private func shareEvent() {
        let postText: String = self.displayText
        let postImage: UIImage = self.renderViewAsImage
        let activityItems = [postText, postImage] as [Any]
        
        let activityController = UIActivityViewController(activityItems: activityItems, applicationActivities: nil)
        activityController.excludedActivityTypes = [.postToTencentWeibo, .postToVimeo, .assignToContact, .markupAsPDF, .openInIBooks, .postToWeibo]
        
        if (UIDevice.current.userInterfaceIdiom == .pad) {
            
            activityController.popoverPresentationController?.sourceView = viewControllerHolder?.view
            activityController.popoverPresentationController?.sourceRect = CGRect(x: (viewControllerHolder?.view?.bounds)!.maxX, y: (viewControllerHolder?.view?.bounds)!.minY, width: 0, height: 0)
            activityController.popoverPresentationController?.permittedArrowDirections = UIPopoverArrowDirection.init(rawValue: 0) //.Down
            viewControllerHolder?.present(activityController, animated: true, completion: nil)
            
        } else {
            
            viewControllerHolder?.present(activityController, animated: true, completion: nil)
        }
        
    }
}

There are a couple utility code snippets that make help out the content view.

This help me access the hosting view controller so I can present the action sheet. I could have wrapped action sheet in SwiftUI but this was way faster to get up and running. I saw this somewhere on Stack Overflow, but I can’t remember where.

struct ViewControllerHolder {
    weak var value: UIViewController?
}

struct ViewControllerKey: EnvironmentKey {
    static var defaultValue: ViewControllerHolder { return ViewControllerHolder(value: UIApplication.shared.windows.first?.rootViewController ) }
}

extension EnvironmentValues {
    var viewController: UIViewController? {
        get { return self[ViewControllerKey.self].value }
        set { self[ViewControllerKey.self].value = newValue }
    }
}

This is what allows me to render the UIView as an image.

extension UIView {

    func renderAsImage() -> UIImage {
        let imageRenderer = UIGraphicsImageRenderer(bounds: bounds)
        if let format = imageRenderer.format as? UIGraphicsImageRendererFormat {
            format.opaque = true
        }
        
        
        let image = imageRenderer.image { context in
            context.cgContext.setAllowsFontSmoothing(true)
            context.cgContext.setShouldSmoothFonts(true)
            context.cgContext.setAllowsAntialiasing(true)
            context.cgContext.setShouldAntialias(true)
            context.cgContext.setAllowsFontSubpixelQuantization(true)
            context.cgContext.setShouldSubpixelQuantizeFonts(true)
            return layer.render(in: context.cgContext)
        }
        return image
    }
    

}

The color view as a flattened version of the one I’m using in the Timeline picker. This control just using a @Binding string variable where the string is the name of a color asset.

struct Colors : Identifiable {
    var id = UUID()
    var colorName: String = "user_purple"
}

extension Colors {
    static func all() -> [Colors] {
        
        // These strings reference named colors in the asset catelog
        return [
            Colors(colorName: "user_purple"),
            Colors(colorName: "user_light_blue"),
            Colors(colorName: "user_cyan"),
            Colors(colorName: "user_green"),
            Colors(colorName: "user_blue"),
            
            Colors(colorName: "user_red"),
            Colors(colorName: "user_orange"),
            Colors(colorName: "user_yellow"),
            Colors(colorName: "user_brown"),
            Colors(colorName: "user_grey"),
        ]
    }
}

struct ColorPicker: View {
    
    @Binding var selectedColor : String
    
    let colors = Colors.all()
    
    var body: some View {
            
        return
            
            VStack(alignment: .leading) {
                    
                HStack(alignment: .top) {
                    
                    ForEach(self.colors) { radColor in
                        
                        ZStack {
                            
                            // Add a selection circle to the current color
                            
                            if (radColor.colorName == self.selectedColor) {
                                Circle()
                                    .stroke(Color.gray, lineWidth: 3)
                                    .frame(width: 40, height: 40, alignment: .center)
                            }
                            
                            Button(action: {self.selectedColor = radColor.colorName}) {
                                Circle()
                                    .foregroundColor(Color(radColor.colorName))
                                    .frame(width: 32, height: 32, alignment: .center)
                                    .padding(4)
                            }
                            
                        } // END ZStack
                        
                    } // END ForEach
                    
                } // END HStack
                    
            } // END VStack
                
        .padding()
             
    }
}

I’m sure there are better ways to handle a feature like this, but this approach is fine for now. There are a number of changes I’d like to make in future versions.

  • Dynamic text / auto-layout. Currently the text views are manually positioned and sized around a simple layout, but this does not allow for different fonts and sizes.
  • It might be interesting to offer some gradients in addition to the colors
  • I’d like to add some decorations such as patterns or simple images that can make the images more interesting.

SwiftUI Modal Badness

I’ve had a longstanding issue in Retrospective Timelines. I have a simple EditButton on the list of timelines in my app. This button toggles the list into edit mode where the user can perform several actions.

  1. Reorder timelines
  2. Delete a timeline
  3. Tap a button to open the timeline in a modal view so they can edit it.

Timeline List Edit Mode

The issue was that when I closed the modal view with a button, sometimes the EditButton on the list view would stop working. Looking at the view hierarchy in debug mode I could see that a containing object for the button was way out of position, and thus the button would stop working. I could sometimes get it snap back into position by swiping up and down on the list, but that is not really a viable solution. I needed a way to solve this.

This is a video I send to Apple as part of the feedback report (FB7265174). You can see the out-of-place container object.

I was once again banging my head on this issue today, trying to see if I could come up with a solution. While working through some options, I noticed that this only happed to SwiftUI Buttons, not to other types of objects in the navigation bar. I made a simple Text view and gave it an onTagGuesture modifier with a call to toggle the edit mode.

Text(self.listEditMode.isEditing ? "Done" : "Edit")
    .onTapGesture {
        self.listEditMode.isEditing ? .inactive : .active
    }

This worked, but I lost the nice animations that SwiftUI was taking care of for me. I came across this post on StackOverflow and modified my view to use this bool version of Edit Mode instead of the the default one that SwiftUI provides. Here is a rough approximation

struct ContentView: View {

    @State var isEditing = false

    var names = ["Item 1", "Item 2", "Item 3"]

    var body: some View {
        NavigationView {
            VStack {
                List(names, id: \.self) { name in
                    Text(name)
                }
                .environment(\.editMode, .constant(self.isEditing ? EditMode.active : EditMode.inactive)).animation(Animation.spring())
                
            }
            .navigationBarItems(trailing: trailingButton)
        }
    }
    
    private var trailingButton: some View {
        
        Text(self.isEditing == true ? "Done" : "Edit")
            .contentShape(Rectangle())
            .frame(height: 44, alignment: .trailing)
            .onTapGesture {
                self.$isEditing.wrappedValue.toggle()
        }

    }
}

This sample code strips away all of the additional logic from my Timeline List view, but it shows how to implement the alternative version of the edit button as a Text view. Finally I have a reliable way to toggle edit mode in SwiftUI.

User Interface Update

I’m making progress on the user interface for the app. Most of my time has been spent on making the Event List views look the way I want. I’ve also updated the Event Edit view with a new timeline picker and a long form text view for editing notes.

Timeline and Event List views side by side. I Added a new top level report called Ongoing that will show all ongoing records from active timelines.


Report list views. These contain an extra timeline element. These list views show data from all active timelines, active meaning “not archived”.


Timeline to Event List to Event Edit.


Event Edit View now contains a long form text view. This uses a UITextView wrapped in SwiftUI. It’s far from perfect but it’s good enough, at least until Apple ships a SwiftUI multi-line text field.


Event Edit View now contains a custom timeline picker.

SwiftUI – A note about onAppear()

This morning I made a custom version of a picker for the events form. I needed a way for events to select a different timeline. The default picker in SwiftUI had some limitations so I set out to make my own. The only main difference is that it uses a sectioned list with timelines sorted into non-archived and archived sections. I made a binding variable to pass in the selected timeline. This allowed me to update the selected timeline from the picker view, then just close it when done.

struct TimelinePicker: View {

    @Binding var selectedTimeline: Timeline?
    @ObservedObject var dataSource = RADDataSource<Timeline>()
    @Environment(\.presentationMode) var presentationMode: Binding<PresentationMode>

    var body : some View {
        List {

            ForEach(dataSource.loadSectionedDataSource(sort: self.getSort(), predicate: nil, sectionKeyPathName: "isArchived"), id: \.name) { listSection in

                Section(header:

                    HStack {
                        Text(listSection.name == "1" ? "Archived Timelines" : "Timelines")
                    }
                    .font(.system(.headline, design: .rounded))

                ) {
                     ForEach(self.dataSource.objects(inSection: listSection)) { timeline in
                        
                        Button(action: ({
                            self.selectedTimeline = timeline
                            self.presentationMode.wrappedValue.dismiss()
                        })) {
                            
                            HStack {
                                TimelineListRowView(timeline: timeline)
                                .foregroundColor(.primary)
                                Spacer()
                                if(self.selectedTimeline == timeline) {
                                    Image(systemName: "checkmark")
                                }
                            }
                            
                        }
                    }
                }
            }
        }
    }

    public func getSort() -> [NSSortDescriptor] {
        ...
    }
}

It just would not work though. I tried several ways of passing in the timeline, binding, observed object, environment object, etc. Nothing would work. Every time I closed the picker the initial value remained set on the events form.

Turns out that’s because I’m an idiot. I was using a call to onAppear to load data from the event record into a View Model class. This was running every time I closed the timeline picker. I would set the initial value, go to the picker, select a new value, and then close the picker which would call onAppear to set the initial value again.

The fix for this was super simple. I just added a new state variable called onAppearCalled and initialized it to false. Then I could check this condition in the function that populates my form when onAppear is called, setting it to true when it is called.

func populateOnAppear() {
    if(self.onAppearCalled == false) {
        ...
        self.onAppearCalled = true
    }
}

My Timeline Picker

Setting up CloudKit with Core Data

This is something I keep forgetting how to do so I’m posting this as a reference for future me and anyone else that finds it helpful.

These instructions assume you have created a new Xcode project with these options.

  • Language: Swift
  • User Interface: SwiftUI
  • Check the box for “Use Core Data”
  • Check the box for “Use CloudKit

This will create some boiler plate Core Data code in the AppDelegate and SceneDelegate.

In the SceneDelete, add this line after creating the context.

context.automaticallyMergesChangesFromParent = true

In Project Settings, select the target and add some entitlements.

  1. iCloud – Add this entitlement and select the “CloudKit” checkbox. Use the default container for your project or select one from the list if you have already created one.
  2. Background Modes – Add this entitlement and select the checkbox for “Remote notifications

That’s it really. From here you can start to work with SwiftUI views by adding the context into the environment for the initial view.

let contentView = ContentView().environment(\.managedObjectContext, context)

Event Date List Row

I’ve been spending a lot of time today thinking about the row for the list of event dates. This has been something I’ve been sort of stuck on for the last couple of weeks. This list is a bit tricky because the data on it is not strictly events. The rows represent date records related to events. All events have at least one date (the single date or start date field) but they can optionals have an end date to form a date range. The ending date can be set to a specific date or set to an “ongoing” status where the event will show up as something that hasn’t ended yet.

For a couple of weeks I was including the ongoing date rows in this list, sorted by the current date. I decided to omit them for now. They always looked a bit out of place to me.

Here are several versions of a new design for these rows. I think I’m going to use the 6th and last option. I removed the circle indicators from the row entirely. The capsule around one of the dates indicates that the date is the one for the row. For example: You can see two rows for the event called “A date range”. It shows up in the list of sorted dates for both its start and end dates. I also decided to make everything in the list content region use the selected color for the timeline, in this case red.

App Icon Drafts

I spent a little time this evening playing with app icon ideas. This is the one I like the most so far.

Update: 2019.10.30 I made my first round of revisions of these. I added some gradients and I replaced the white color with a light grey and I think looks pretty nice.

1 2 3 4 5