Consuming Unity's WebView API

or how I ended up with a one-way ticket on the Unity pain train

In lieu of an update this week, I'd like to talk in depth about one of the more technically challenging parts of Easy Feedback: in-editor authentication via WebView.

Before launching early access, Easy Feedback was in somewhat of a "closed beta" state. I had given the asset to fellow game developer and streamer, Qazi, to use in his latest game, Shotgun Farmers. At that time, the only way to authenticate with Trello was to navigate to the API homepage in a browser, and request an application key and secret. Qazi then had to edit a script that held the Trello API key and secret as constant strings (aptly named TrelloAPIConfig), adding his key and secret. This was all a holdover from when the feedback form was originally written for my game, Noah and The Quest to Turn on The Light. At the time, I hadn't planned to release the form as a standalone asset, so I didn't put much thought into reuse.

Before launching the public alpha, one of the first priorities was abstracting users away from the Trello API as much as possible. Ideally, I wanted to keep the users within the editor at all times. With a constant application key for all of Easy Feedback, I can direct users to a URL where they will be required to log in and give Easy Feedback permission to access their account. After they give permission, users are given a single token that they can copy and paste into the Easy Feedback script. Easy enough, now we just need to show all that in the editor.

This is where things get tricky. The WebView API (as seen in action in the Asset Store window), is not publicly accessible. In fact, it's an internal sealed class, and there's no documentation on it at all. But, I was determined. If the Asset Store could display web content, I sure could too. Accessing non-public types, methods, and fields in C# isn't impossible, but it's also not easy.

Thanks to reflection, if I expect a type, method, or field to exist, I can find it, execute it, or access and modify it, whether public, private, sealed, or internal. The downside to this technique is that I have to know the exact definition of the type ahead of time, and any mistakes result in frustratingly vague runtime exceptions. This is where Matt Rix's Unity Decompiled repository came in very handy. Not could I reference the full definition of the WebView class, but I could also reference the AssetStoreWindow class to figure out how to properly use WebView.

With all that covered, let's talk about how the authentication window implementation works.

Initializing the EditorWindow

When the window is first initialized, we have to do a few things:

First, we get the HostView for the window. This is a private field in EditorWindow, named "m_Parent." We'll need this later, when initializing the WebView.

FieldInfo parentInfo = typeof(WebWindow).GetField("m_Parent", ALL_FLAGS);
guiViewParent = parentInfo.GetValue(this);

Next, we search the application assemblies for the WebView and GUIClip types. Finding the types is pretty simple, we just search through each assembly in the current application domain until we find a type that matches a given string name, then return that type.

private Type getTypeFromAssemblies(string typeName)
{
    Assembly[] assemblies = AppDomain.CurrentDomain.GetAssemblies();
    foreach (Assembly assembly in assemblies)
    {
        Type[] types = assembly.GetTypes();
        foreach (Type type in types)
        {
            if (type.Name.Equals(typeName, StringComparison.CurrentCultureIgnoreCase)
                || type.Name.Contains('+' + typeName))
                return type;
        }
    }
    Debug.LogWarning("Could not find type \"" + typeName + "\" in assemblies.");
    return null;
}

We only need the static "Unclip" function from GUIClip, so we go ahead and reflect out the method information from the type. We pass in the Rect type when calling GetMethod on the type, as GUIClip contains multiple definitions for Unclip, and we want the one that returns a Rect.

// get GUIClip.Unclip
unclipMethod = guiClipType.GetMethod("Unclip", new[] { typeof(Rect) });

Initializing the WebView

Disclaimer: because of the lack of documentation, all the code from here on out is the result of heavy reference of AssetStoreWindow, and a lot of trial and error. I make no promises that this is the correct, most efficient, or even most stable way to render a WebView. By continuing, the reader assumes all liability for all emotional and physical damage as a result of reading past this point.

private void initWebView()
{
    // dispose and destroy webView
    if ((ScriptableObject)webView)
    {
        DestroyImmediate((ScriptableObject)webView);
        webView = null;
    }
    // create the webview
    webView = ScriptableObject.CreateInstance(webViewType);
    webViewType.GetMethod("InitWebView").Invoke(webView,
        new object[] { guiViewParent, (int)position.x, (int)position.y, (int)position.width, (int)position.height, false });
    // load url
    webViewType.GetMethod("SetDelegateObject").Invoke(webView, new object[] { this });
    loadURLMethod = webViewType.GetMethod("LoadURL");
    loadURLMethod.Invoke(webView, new object[] { URL });
    // get methods from webview
    showMethod = webViewType.GetMethod("Show");
    setFocusMethod = webViewType.GetMethod("SetFocus");
    setSizeAndPositionMethod = webViewType.GetMethod("SetSizeAndPosition");
    reloadMethod = webViewType.GetMethod("Reload");
    executeJSMethod = webViewType.GetMethod("ExecuteJavascript");
}

First, we check if we already have a reference to an existing WebView object. If so, we destroy it so that we can initialize a new one. Next, we instantiate the WebView with ScriptableObject.CreateInstance, and invoke the InitWebView function on our newly created object. This is a good time to talk about why Unity Decompiled is so helpful: When using reflection to invoke a method on an object, you need to know the method's full signature. All parameters are passed in as an array of objects that must all be present, of the correct type and in the correct order, or the runtime gods will smite you with incredibly unhelpful exceptions or editor crashes. Referencing Unity Decompiled is the only tool we have to fight the runtime gods. This is where we pass in the GUIView we retrieved in init.

// create the webview
webView = ScriptableObject.CreateInstance(webViewType);
webViewType.GetMethod("InitWebView").Invoke(webView,
    new object[] { guiViewParent, (int)position.x, (int)position.y, (int)position.width, (int)position.height, false });

Next, we invoke the SetDelegateObject method on the WebView, and pass in the current EditorWindow instance. Then, we reflect out the LoadURL method, invoke it with the current URL as the only parameter, and save the method info for later use.

// load url
webViewType.GetMethod("SetDelegateObject").Invoke(webView, new object[] { this });
loadURLMethod = webViewType.GetMethod("LoadURL");
loadURLMethod.Invoke(webView, new object[] { URL });

Finally, we reflect some helpful methods out of WebView, and save them for future use. For each of the methods, we'll also write helpful wrapper methods for quick and clean invocation.

// get methods from webview
showMethod = webViewType.GetMethod("Show");
setFocusMethod = webViewType.GetMethod("SetFocus");
setSizeAndPositionMethod = webViewType.GetMethod("SetSizeAndPosition");
reloadMethod = webViewType.GetMethod("Reload");
executeJSMethod = webViewType.GetMethod("ExecuteJavascript");

Rendering the WebView

If you thought we were flying pretty loose and wild before, strap in.

private void OnGUI()
{
    if (unclipMethod != null)
    {
        webViewRect = (Rect)unclipMethod.Invoke(null, new object[]
        {
            new Rect(0f, 0f, base.position.width, base.position.height)
        });
    }
    else
    {
        init();
    }
    //Rect webViewRect = new Rect(0f, 0f, base.position.width, base.position.height);
    if (webView == null)
    {
        init();
    }
    else if (Event.current.type == EventType.Repaint)
    {
        try
        {
            //doGUIMethod.Invoke(webView, new object[] { webViewRect });
            setSizeAndPositionMethod.Invoke(webView, new object[]
            {
                (int)webViewRect.x,
                (int)webViewRect.y,
                (int)webViewRect.width,
                (int)webViewRect.height
            });
            // set white background
            executeJS("if(document.getElementsByTagName('body')[0].style.backgroundColor === '') { document.getElementsByTagName('body')[0].style.backgroundColor = 'white'; }");
        }
        catch (TargetInvocationException e)
        {
            // web view was disposed, but not by us :(
            webView = null;
            DestroyImmediate(this);
        }
    }
}

I'd like to address a reoccurring trend in this block. Sometimes, OnGUI may be called before init, or references may be cleaned up by some mysterious outside force. To counter this, we take care to check that things are not null before trying to use them. If they are we just initialize the window again.

The first thing we do in OnGUI is invoke the Unclip method we retrieved in init. To be completely honest, I'm not completely sure what this does. But the Asset Store window does it, and it works.

if (unclipMethod != null)
{
    webViewRect = (Rect)unclipMethod.Invoke(null, new object[]
    {
        new Rect(0f, 0f, base.position.width, base.position.height)
    });
}
else
{
    init();
}

If this call to OnGUI is the repaint pass, we attempt to Invoke SetSizeAndPosition on the WebView, setting it to the size and position of the current editor window. Sometimes, our WebView object may still end up null here. In that case, we just go ahead and kill the window. This was more of an issue in development, and I haven't been able to reproduce it, but I'd rather be safe than sorry.

if (webView == null)
{
    init();
}
else if (Event.current.type == EventType.Repaint)
{
    try
    {
        //doGUIMethod.Invoke(webView, new object[] { webViewRect });
        setSizeAndPositionMethod.Invoke(webView, new object[]
        {
            (int)webViewRect.x,
            (int)webViewRect.y,
            (int)webViewRect.width,
            (int)webViewRect.height
        });
        // set white background
        executeJS("if(document.getElementsByTagName('body')[0].style.backgroundColor === '') { document.getElementsByTagName('body')[0].style.backgroundColor = 'white'; }");
    }
    catch (TargetInvocationException e)
    {
        // web view was disposed, but not by us :(
        webView = null;
        DestroyImmediate(this);
    }
}

An extra fun feature of the WebView is that any webpage without a defined background color will render with a transparent background. This isn't an issue of course, because the Trello API pages all define a background color, right?

Heck.

From what I can tell, the WebView mostly takes over and handles rendering within its delegate editor window on its own, and simply rendering a blank white texture in the background doesn't work. Great.

But, it's 2017, and no web problem is solved in 2017 without JavaScript. Luckily for us, WebView has an ExecuteJS method lying around. We call our wrapper function, and execute JavaScript that sets the page's background as white if it does not already have a defined background color. We do this on every OnGUI call.

Excuse me, I need a moment.

// set white background
executeJS("if(document.getElementsByTagName('body')[0].style.backgroundColor === '') { document.getElementsByTagName('body')[0].style.backgroundColor = 'white'; }");

OnFocus and OnLostFocus

Whenever the window gains or loses focus, we invoke the SetFocus method on the WebView, and pass in the focus state as a bool. If the window is currently being focused, we also invoke SetHostView on the WebView and pass in the editor window's GUIView, then invoke the Show method on the WebView. Again, the Asset Store does this, so we're doing it.

private void OnFocus()
{
    setFocus(true);
}
private void OnLostFocus()
{
    setFocus(false);
}
private void setFocus(bool focus)
{
    if (webView != null)
    {
        try
        {
            if (focus)
            {
                webViewType.GetMethod("SetHostView").Invoke(webView, new object[] { guiViewParent });
                showMethod.Invoke(webView, null);
            }
            setFocusMethod.Invoke(webView, new object[] { focus });
        }
        catch (TargetInvocationException e)
        {
            // web view was disposed before we expected it :(
            webView = null;
            DestroyImmediate(this);
        }
    }
}

OnDestroy

Don't worry, this is all almost over. Finally, when the editor window is destroyed, we invoke DestroyWebView on the WebWindow.

private void OnDestroy()
{
    // destroy web view
    if (webView != null)
    {
        webViewType.GetMethod("DestroyWebView", ALL_FLAGS).Invoke(webView, null);
        webView = null;
    }
}

Final thoughts

If you enjoy the Unity Editor crashing every time your scripts recompile being a primary element of your debugging workflow, I highly recommend trying out implementing an undocumented internal API.

There's a lot of mess and inefficiency in this script, but it's functional for the time being, and the end result really does improve the authentication flow. When I have more core features implemented, I'll be coming back and cleaning things up before full release.