One optimal way of driving UMG (Unreal UI)

So you're a Unity developer switching to Unreal. You hop into this new UI system called "UMG", and at first glance, it seems alright? So you go to make your 9-slice window image base so you can begin and-

You've already doomed yourself. You just don't know it yet. Your next week is full of screams.

Now, before I continue, let's set some ground truths:

1.) I don't claim to be a UMG expert

2.) Everyone's got different ways of leveraging UMG, and if the system supports it well, great, that's a fine way of driving it

3.) It's more fun to write in a highly opinionated way

and having gotten that out of the way, here's how you should drive UMG (and why)

We're going to be working at 4k

"But why 4k? That's like 6% of all gamers according to steam!" you might think. I have three fantastic reasons why we're going to be working at 4k:

1.) Consoles. Consoles are very very often rendering at 4k these days. Not because anyone normal cares about 4k, but because when you buy a TV at Costco at this point, it's 4k. You can't escape it. It isn't particularly GOOD 4k with HDR, but all that stuff is there. So we need to design with that in mind.

2.) Unreal is very good at automatically scaling things DOWN. If you hand it 4k, it will just shrug and hand you LOD'ed versions of everything for lower quality settings. You won't even have to think about it. It is, however, not good at automatically letting you scale quality UP. Trying to stuff in higher-res versions of HUD images that you swap in depending on settings or, whatever? Fucking nightmare zone. So we always work at maximum display res we're ever likely to support, and scale DOWN.

3.) Unreal is just bad at pixel precision. You're going to make a perfectly square 100x100 square in your UI, and you're gonna look at the file, and the resolution on that file will read 101x100. You certainly could scream about this, and fight for pixel precision. It is, technically, possible to get a pipeline working for pixel precise UI.

But that road is paved with screams.

My road? It's paved with that nice pea gravel and there's lil water fixtures installed along it. This is the path I found through the field of rakes, and I promise it goes somewhere good, but the whole reason it works is because you roll with it. It lets you work around lots of places where things don't quite meet up perfectly. I'm assuming this is because whoever made UMG just didn't care about pixel precision (more on that later, there's tons of layers to why), but the point is that we design our pipeline such that we don't have to care either.

or you take the path paved in screams. I'm not your mom. But I sure ain't walking down there with you. Which brings us to the first cause of why UMG is so bad at pixel precision, in fact!

Use HorizontalBox and VerticalBox heavily

UMG wants you to build things out of boxes and reserved space, that essentially scale themselves. It wants you to space your central menu column by stacking it inside a series of horizontal and vertical boxes. It enforces this by how relatively few widgets allow for more than 1 child.

When you need to go wide with children, you want to use a Panel or Overlay. As an aside here (EDIT: as in I've just been told), you PROBABLY actually want to use Overlay widgets, because Panels add draw calls. Again though, the flow will work better if you think more in terms of stacked horizontal and vertical boxes. Need to inject size? Use a SizeBox. You will end up with SO MANY stacks that look like this:

Use "Wrap With" and "Replace This" to manage the hierarchy

"but how do I replace things and change hierarchy around if everything can only have one child!" I hear you crying (I cried so, so much). Easy! Just right click, and click these obscure menu options that no other system has, and there you go! Of course! SIGH.

Seriously tho, that's what you do. "Wrap With" is used to inject a parent, and "Replace With" is sort of doing the whipping the tablecloth out and slipping a new one in without moving the plates, trick.

Of special note here is "Replace With" and then selecting "Child". That's how you remove things from hierarchy. Right click the thing you want to remove, Replace With, Child, and its child zooops up and the right clicked thing is gone.

Leveraging these, and panels, is thus how you re-arrange things. Right Click, Wrap With, Panel or Overlay, and now you have a workspace at that level of hierarchy to move shit around in. When you're done, Right Click the temporary panel, Replace With, Child, and now you're back to normal.

Once you've got the basic widgets figured out, and you get tired of programmer greybox UI, you're going to realize most things are an Image. Images are your backgrounds, buttons are awash with textures, and so on. So moving into how we create assets for this beast, and first all,

Ditch 9-slice

First of all, ditch 9-slice. By which I mean, the kind of window stretching that preserves the corners and tries to preserve the edges but stretches the middle, and. No. Really. I'm serious. Unreal's 9-slice is terrible. It's got some very specific use cases we'll get to later, but if you use it like you would Unity's, you will scream. Instead,

Export all your window components to target size

By which I mean, in your layout editor, actually make a workspace sized to your target resolution. Which should be 4k, btw, NOT 1080p, but we'll get to why in a sec. Another big gotcha is that you need to set your DPI to 96. Unreal uses 96! This probably isn't the default in your tool!

So you've got your big 4k display-sized surface, and now you're going to lay your UI out exactly as it would appear in-game. Instead of making one window skin that you use everywhere, you make every single window as a separate element. Which sounds wild to a programmer, but to a designer? Uh, Affinity and similar already have tools for that kind of thing. Use 'em. Just shift your thinking from that being done at engine level to it being done in your editing layer.

Look, I know what you're thinking, "but memory usage" but it's 2024. Come on now. Your entire UI's still going to take up less space than a single AAA character's texture stack. Besides, this has some HUGE advantages, namely: you can actually texture your windows! You can put ripples in them. You can wear the edges. Also even fancier stuff if you play with materials (we'll cover that in a sec).

As a quick aside for tooling, I like Affinity Designer 2 for this job, but I've heard Inkskape is also rad. To that end, in Affinity, you end up kicking over to Export persona, clicking the layers tab, selecting everything that you want welded into a single texture, and defining that as a "slice", which in Affinity is this tiiinnnyy little button in the lower right of the Layers panel but only in Export persona. Anyways, now when you hit export, you get a nice single PNG per widget. Or maybe two files if you separate the mask out, that's another thing we'll talk about in a bit.

Anyways! Now that you have a single texture, sized to a particular size for the window, it is MUCH easier to use in Unreal. This is how it's meant to work. Go make an Image widget in UMG, point it at the texture, tell it how big it is in pixels, and it just works. Done. ezpz.

Oh and speaking of target resolution / number of pixels,

Set Unreal's UMG editor up to operate in 4k coordinates

By default, UMG / HUD will scale in a weird way. I'm not going to express an opinion on it, since there's use cases, it's just uhhhhh. Anyways. Go project settings, engine - user interface, set DPI scale rule to Scale To Fit, then set design screen size to 3840x2160. Now UMG will be working in 4k coordinates, and will scale that 4k space to any display resolution, so it'll work with your 4k assets. More importantly, now the game won't look like garbage at 1080p due to subpixel sampling errors. It'll probably also look fine at 4k, just because 4k displays tend to have way higher pixel density than 1080p screens, so black edges are less noticeable.

Now that setting we changed, Scale To Fit? Not actually necessary for this. This can also be done by manually adjusting the dpi scale curve thingy so that 1.0 is at 4k resolution, but. Ewww? Seriously. Eww. Just use Scale To Fit.

What's all this talk about black edges? subpixels?

Right, so, black edges are due to transparent pixel color. You might be wondering why this blog is so light on images, and it's because, the mere act of encoding a screenshot all but hides the error, but trust me. You'll see it. If you design your UI at 1080p res, and then place that over in UMG using 1080p-scaled coordinates, and display that on a 1080p screen, you'll see this nasty black halo edge around things.

It's caused because, for most transparent image encodings (PNG but also everything else really), transparent pixels are written out as black. You're not supposed to see those black pixels, because they're transparent, but sampling isn't pixel-precise. Any time the renderer goes to sample a pixel on the edge of your widget, it's actually slightly offset from the precise center of the pixel, and so you end up with an average of all neighbor colors. Including the black behind the transparent pixel. Hence, black edges.

There's a couple of fixes here!

• Firstly, the simple act of shifting to 4k (which you already did), hides almost all of this. It really is just hiding it, it doesn't fix anything, but it's a pretty good dodge. You're essentially gambling that pixel density on 4k screens, a/o viewing distances, will always be sufficient that single pixel edges don't contribute much. Start here. If it's good enough, no need to do the rest. This probably at least gets you through pre-production.

• If this still isn't enough, the next simplest port of call is a kind of silly material hack:

• All you're doing here is saying "if a pixel is too black, fuck it, it's transparent." That's it. I can not express how stupid this is. And yet it works pretty well! Note that I haven't included vertex color or anything yet, which you'd need to do if you wanted this to work with UMG tinting. (I think? again haven't tested but I think it's vertex color, ANYWAYS whatever)

  • Side node, making a custom UI material for all your widgets, as in instead of pointing the UMG Image at your texture file directly, you point it at a Material that's nothing but a sampler for said texture? Will also reduce the edge issue. EVEN IF YOU DON'T DO THIS TRICK. No, I have no idea why. The default UMG material, apparently, does some nasty fucking shit I can't make sense of. So it's pretty good practice to wrap your UI textures in materials anyways, just, in general. Again, I'll get to why materials are cool in a sec.

• Now if this still isn't good enough, the next options are equally nerdy and involved just for different skill-sets.

  • If you're a programmer, you can build or find (there's a python script I found) a script that does a manual color bleed on all your UI PNGs before import. The logic is simple, you're just going through and replacing the color of transparent pixels with whatever their nearest not-invisible color value is. Hence, bleed, you bleed the edges out. By the way, no, the Bleed option in Affinity or similar doesn't do this, I know, but that isn't what it's for, that's print Bleed, different concept. Anyways, once you've color-bled the black out, the borders just go away like magic.

  • If you're an artist or don't want to hack at your files, your other option is to stop relying on transparent file formats. Instead, you're going to export two files per UI widget: the color, and the mask. Now incorporate the two in a material so trivial I don't even need to show it (hint: sample both textures and just jam the mask's R channel into Opacity, and you're done), and tada, you have full, precise, artistic control of the issue. Have fun. You can now paint your own bleed, too, since there are no actually transparent pixels, only diffuse and mask layers. This method will either seem much better, or painfully fiddly, depending on how your brain works.

You keep mentioning how Materials let you do cool things?

So by now you're building your UI to-size in your designer tool, exporting a bunch of PNGs, importing them

Everything so far is just me having broken ground through the rake field, I stepped on all the rakes, and I'm showing you the path I took so you don't step on any rakes and know you'll get somewhere stable. Everything past this is a bit looser, and involves speculating on why the field of rakes was even there to begin with.

First of all, to do any cool FX in a UI, you really want the pixels in the texture to map 1:1 to the pixels being displayed. Like, imagine if I want to make the window do a ripple, or a dissolve. If I was working with a 9-slice widget, that'd mean that the entire center of the window was massively stretched, so I'd go to do my ripple or dissolve, and the effect would look nasty as hell, all stretched over the middle.

But ah hah! Remember how all our UI textures were made to size? We don't have that problem now! I have 1:1 pixels. I can ripple the whole window, I can dissolve it, I can shoot it full of bullet holes, any of that wild stuff that hasn't really been en vogue since the early 10's but you still see in AAA shooter UIs? You're set up for ALL of that.

Now, why is that?

So! UMG didn't get created in a vacuum. UMG was born back in an era where another competing UI standard, Scaleform, reigned supreme. Scaleform was basically just Flash. ActionScript. That thing that's been dead for a decade. Suffices to say it was like having a vector drawing program in your HUD, you could do just about anything, so it was great for FX stuff and terrible for anyone who just wanted to make a fucking window and be done with it.

Now step back, and look at what we've gained by doing our UI in this precise way. What we've got isn't precisely capable of what Scaleform could do, but we're at least sort of close! Because again, 1:1 pixels, so a sufficiently invested tech artist can do just about anything in here that they want. That makes this a good replacement for Scaleform. Now wind a decade ahead, when we don't really make UIs where we blur the HUD and shoot it full of bullet holes, and- well, the system is still from that era, even if we don't think to use it that way anymore.

This may not help you actually like UMG's limitations, but I find it at least helps me understand why it is how it is.

This is also a good time to talk about 9-slice. 9-slice, in Unreal terms, is itself a Material, so pro users who want good 9-slice probably don't even use the built in stretching mode, because it's not great. They just write their own material. The built-in 9-slice is really only good for the narrow case of like, "relatively simple inventory or popup screen that I mostly display at one size, but want some variance in case it's a bit bigger or a bit smaller". It still won't really work right unless you hand it a base widget texture that you've designed be the full-size window. Also, it stretches the edges and middle, it doesn't tile, meaning it's only really good for 20% either way on size before the distortion will get too bad. So again. It isn't that the built-in 9-slice is bad, it's that they started (scroll up), and got to the end of that process, and went "ok, and now we need the occasional slightly-scaling window, let's make a system for that based on all these other assumptions we already made."

Make more sense now? Hopefully it helps contextualize things a bit? Hopefully!

What's Slate?

Slate is the underlying system (API?) under UMG. It's much, much less friendly. Yes I know how that sounds. It's what you'll have to interact with if you dig heavily into modifying the engine UI itself, or other stuff. It's C++, and it's gnarly, and it's entirely out of scope for this blog. I'm only explaining it so that when you see Slate, you can mentally go "oh, the nightmare C++ underlying UMG, right, not my monkeys, not my circus, moving on."