After the excitement and the fun that is the Red Hat Summit, I had some time this week to work with Jason on updating the design. Before I dive into the design process and the mockups, I think it’d be best to do a review of how cgroups work (or at least how I understand them to) so that the rest makes more sense. (And maybe I’ve got some totally incorrect assumptions about cgroups that have resulted in a flawed design, so hopefully my calling out the current understanding might make it easier for you to correct me ).

A designer’s understanding of cgroups via diagram

So cgroups, which are sometimes referred to as containers (I think because a similar Solaris feature, zones, is sometimes called containers) can be used to slice an entire operating system into buckets, similarly to how virtual machines slice up their host system into buckets, but without having to go so far as replicating an entire set of hardware.

So this diagram kind of shows how four of the system resources that cgroups can control – CPU, memory, network, and storage I/O – could be cut into slices that are then combined into two groups – the yellow and the purple one – which make up virtual OSes. Say I gave cgroup #1 (yellow) to Sally, and cgroup #2 (purple) to Joe. Whenever Sally starts a process, you could set it to only run on the CPUs that are members of cgroup #1 (via cpuset), at whatever priority level is set for those CPUs (via cpu). It’ll only be able to use as much memory as was allocated to cgroup #1, only be able to use as much network and I/O bandwidth as cgroup #1 is able to use. When Joe starts a process, because he’s part of cgroup #2, he won’t be using the same CPUs as Sally. He may have more or less memory, I/O, and network bandwidth allocated to him.

It’s kind of / sort of like Joe and Sally are using different computers, on the same operating system. Cool, right?

Cgroups don’t have to go that deep, though. You don’t need to slice across an entire system.

You can have a cgroup that *just* deals with controlling access to the CPU. Or *just* controls memory. Or maybe only deals with two of the four (CPU, Memory, Network, I/O) resources, or just three of the four.

You can have a cgroup that *just* deals with one resource (say CPU), and that group only deals with specific processes. Or users. Or a combination thereof. (More later on that.)

Depending on the resource you’re looking to control and the cgroups module you’re using, you can configure access to that resource in different ways. I believe one of the more common ways of controlling CPU usage via cgroups is to assign ‘shares’ for various groups’ usage of the CPU (using the ‘cpu’ module.) I’m not sure what scale/units these shares are on, but they are relative to each other, so if I give group #1 a weight a 1024, and group #3 a weight of 2048, then group #3 will get scheduled for CPU time twice as much as group #1 will.

I don’t think this share system is particularly intuitive, which is still an open problem in the current draft of the UI design.

Other resources and modules let you control access in different ways. For example, the memory module lets you configure an upper bound of memory usage, I believe via providing the maximum number of bytes of memory that could be used by members of the group.

You can create and apply cgroups to processes and users on-the-fly or on a longer-term / persistent basis. Say some process is running amok and is starving other processes on your system… you can change the process’ cgroup membership on-the-fly to provide it a more limited set of system resources so that other processes on the system can run. However, this change would be temporal and may be based, for example, on a specific pid number that won’t apply if the process is restarted or the system is rebooted. If you’d like more persistent cgroup membership, you can create a set of rules (cgrules.conf). A neat simple thing you could do with cgroup rules, for example, is something mentioned by Linda Wang in her Red Hat Summit talk on cgroups; you could arrange your cgroups rules such that the sshd always gets a dedicated chunk of CPU time so that if a process runs amok on a server system, you still might be able to ssh in remotely to diagnose the problem.

It’s via these rules that you can set up persistent groups on the system. It would be kind of a pain to have to set them up every time a system is rebooted, especially the more processes and users you’re managing, and the more groups you need to create in order to manage them. You can use multiple cgroups modules (for example, cpu, cpuset, mem, net) within a single cgroup, and then write rules to place processes run by particular users and/or user groups into the cgroup, or write rules to place processes matching particular attributes into the group.

Above is a diagram demonstrating a cgroup that restricts only CPU usage for processes matching firefox-*, npviewer.bin run by users that are in the ‘guest’ group or whose usernames are ‘student1′ or ‘student2′. Kind of putting it all together, showing a single group and its resource allocation, and its associations with particular users and processes.

Who would use a UI for this, and why?

Tthe UI design mostly focuses on setting up persistent rules, and doesn’t really allow for on-the-fly cgroups rearrangement of currently-running processes and currently-logged in users. The thinking behind this is that there might be a couple of main reasons you’d be using a UI for cgroups:

• Proactive: Initial system resource allocation planning
• Reactive: In response to a complaint – ‘My processes are getting capped,’ or ‘Such-and-such process isn’t running right.’

Thinking about the way my previous experience with system administrators has typically gone, there’s sadly not usually a lot of time for proactive planning and organization – a majority of time tends to get spent on reacting to client concerns. If someone is calling you up on the phone to tell you that a process they are relying on has gone awry, it seems the most effective way to get them to stop calling you would be to change the rules to make a more persistent change to help them, rather than to just apply a change on the fly and wait for their next phone call. So I think you’d probably prefer to change rules in reaction to a client complaint, not just do an on-the-fly change (although you could.) When you’re being proactive and initially setting up a system, I think you’d not want to do that on-the-fly at all, because you’d have to keep reapplying. It’d be better to craft a set of rules to persist on the system.

So it seems like these two main use case types – proactive resource allocation, and reactive allocation adjustments based on client feedback – are both best served by focusing on rules, so that’s why the UI design only focuses on those.

Maybe there is a case for doing the on-the-fly stuff though. It does make for some pretty awesome demos, though, like the one Bob Kozdemba gave at the Red Hat Summit last Friday, moving multiple copies of a graphics rendering tool between cgroups and vms and changing their configuration on the fly so you could visually see the affect of resource caps on the processes.) You could also use the on-the-fly configuration changes to test out a theory about how you should set the rules.

So, specifically, what kinds of problems could we imagine a user looking to solve with this interface? Our thoughts were that most likely a system administrator would find it most useful:

Under the ‘Proactive’ Category:

• I have a system running a business-critical app that sadly has a memory leak, and I want to make sure that app doesn’t screw up the other apps running on the system. I’d like to put a cap on its memory usage so when it goes down it doesn’t take the rest of the system down with it.
• I have a thin-client lab meant for the use of students in the science department, but students from other departments are allowed access via guest accounts. I’d like to make sure the guest accounts don’t block any of the science department students’ work on the server.
• I’m an administrator at an ISP and we provide virtual machines to end users in different pricing tiers. I’d like to enforce the limits set by our pricing tiers so that customers’ VMs aren’t getting more power than they are paying for and aren’t starving customers that are paying for more power.

Under the ‘Reactive’ Category:

• A student from the science department has called helpdesk to complain that his simulations don’t have very much oomph and are taking far too long to run. I need to figure out what system resource policy might apply to this user, and make modifications to his policy as needed to help him out if possible. (For example, perhaps he was previously outside of the science department and just transferred in, so he’s still running in the guest group.)
• Uh-oh, something’s gone wrong. A business critical process keeps getting OOM’ed! I need to figure out what resource allocation policy applies to it to see if there a rules that are causing the issue / that could be adjusted to get the app running consistently again.
• Our bi-weekly payroll processing application is still going full-steam ahead but it’s Monday morning, the business day is starting, and that server needs to be used for other things. Whoops, that’s never happened before, it usually finishes on Sunday. I need to see what policy applies to it, and modify our rules so that the payroll process tones things down a bit if it hasn’t finished over the weekend.

The mockups

So after a lot of sketching, crossing out, sketching, crossing out, and thinking about the above types of use cases, we thought to break up the main window in this way:

• A users tab: For when you have a specific user/group in mind you’d like to limit, or if you’ve got a specific user on the phone who you’re trying to troubleshoot with;
• A processes tab: For when you have a specific process in mind you’d like to limit, or if you’ve got someone on the phone upset about a particular process you need to help debug;
• A containers tab: For when you need to initially create your cgroups, or if you’d like to tweak the configuration of a particular group, or if you’d like to see how the processes within the group are performing.

So here’s what they look like:

The users tab

The processes tab

The containers tab

Some issues / further work

Here’s a bit of a braindump of where these need more work:

1. So I mentioned in one of the example use cases a payroll application, that could run full steam during the weekend but had to be dialed back during the work week. Well – this UI design doesn’t account for scheduling in the rules, yet.
2. There’s no mockups for the various dialogs needed for adding and modifying rules.
3. It would be cool to do a mockup showing how this might integrate into the GNOME system monitor – maybe an additional filter on it?
4. It would also be cool to have some kind of integration with the user accounts dialog. Maybe it’d show the policy that applied to a user and let you modify it from that dialog?
5. Right now in the containers tab, we had the idea to list the containers by the resource they are managing – so groups involving cpu, cpuset, cpuacct for example would all be listed under a ‘CPU’ category. However, Jason had an idea about showing logical cgroups that span resource types – e.g., you might have a cgroup tuned for DB usage with different settings for CPU in combination with memory & I/O.
6. The users and processes tab doesn’t really let you monitor resource usage per user / per process. Is that needed? Not sure.
7. Units – shares vs upper limits vs… the actual numbers you’re configuring for each rule need to be fleshed out. If cpu / cpuset / cpuacct for example can be used in conjunction, how would that be shown in the UI?
8. What about a mechanism to compare effective resource allocation vs actual resource usage? So you could see, ‘hey, this process is starved, maybe we should consider upping it,’ or ‘hey, this user isn’t really coming anywhere near their limits, should we dial them down?’ An earlier draft of the mockups showed graphs of each for comparison.
9. (Edit: idea from Bill Nottingham) Support multiple systems in one UI over the network

The wiki page for this design (including Inkscape SVG sources) is here:
http://fedoraproject.org/wiki/Design/CGroupsUI

Feedback

Is this nuts? Does it make sense? Is this the wrong approach? Do you use cgroups? Would this be a useful tool, or does it suck?

I hope you’ll let us know.

From what I’ve gathered about this UI:

• It’s fairly old.
• It’s grown organically, with new options and features added on piecemeal without an overall design vision.
• It exists in firstboot too, under the ‘network login’ button.
• It allows you check off as many and whatever identity and authentication methods you desire, even if the combinations make no sense.

That last point leads me to classify it as a good example of a box of chocolates GUI, meaning ‘you never know what you’re going to get.’ Configuring isn’t really a task that most people really have as a life goal nor is it something generally considered fun (it’s the cool stuff the configuration eventually enables you to do that’s fun!), so I think configuration / administration UIs like this often degrade to the ‘box of chocolates’ state. Authconfig-gtk will try each combination possible from your selections, trying each little chocolate, er, moving on from failure until it hits one that ends up being cherry creme-filled, er, actually works.

The catalyst for revisiting the UI is a cool new technology called SSSD that folks such as Stephen Gallagher and Dmitri Pal are working on. SSSD stands for ‘system security services daemon.’ It is going to unify identity and authentication access and enable features such as offline login and caching support for centrally-managed system logins (e.g., you don’t need to be on your corporate VPN to be able to log into your laptop which uses an LDAP account on that VPN.) The challenges:

• SSSD does not yet support all the protocols that the legacy stack does (for example, Winbind) and won’t for Fedora 13.
• SSSD enables you to do things the legacy stack also supports.
• There are still users who need to use the legacy stack, so it’s not yet retire-able.
• There are advantages to using SSSD’s currently-available functionality, so it’s important to provide UI access to it so folks can try it out.
• Multiple ways of doing the same thing via very different frameworks has a high risk of really confusing users.

At first I struggled a little bit with how to handle the situation. For Fedora 12, a panel specifically for SSSD was simply grafted onto the existing UI, and the model was a little similar to how the GNOME Shell preview works in the Desktop Effects panel of Fedora – it’s this new technology you can ‘enable’ or ‘disable’ in a modal manner:

For Fedora 13, the path Stephen, Dmitri, and Tomas Mraz considered at first was to build a shiny new and fresh UI to handle SSSD-specific configuration, and keep the legacy UI as-is alongside it, with the plan to eventually retire it. The first set of mockups I put together along this path exposed some problems that Matthias helped identify as being too problematic:

• The user has a task they want to do. We complicate their being able to do it by having to ask them questions along a line so we can figure out what technology they intend to use so we know whether or not to present them with the legacy UI or the new UI. This would complicate firstboot terribly. It’s essentially allowing the implementation to surface on the UI – inside-out UI is no good.
• Making every single user choose what they want in every case is more difficult than giving the users the most common / useful configuration by default and requiring only some (not all, and hopefully a minority) of users to make other choices and change the defaults. I think the latter is a design pattern we should strive to follow in Fedora.
• Having two UIs means you might need two menu items – we have enough configuration items in the menus already, and how are users to know which UI they need to go into to accomplish what they need done? It’s not like there would have been a clean split between the two – the legacy UI manages both local and centrally-managed accounts while the new UI manages some centrally-managed account types only.
• Having the legacy UI pop up in firstboot means having three or four levels deep of windows popping up – firstboot isn’t a desktop environment and it’s awkward to manage windows in a limited environment like that. Yes, it’s like that now, but it doesn’t mean it’s the right way to do it.

Matthias really helped me on the path towards a design that I feel is a whole lot cleaner and simpler to use. Simply put, the new approach here is to take the existing UI, bring it up to speed and give it a clean overall design, and integrate the new SSSD technology into it without it being so painfully apparent which stack is doing what by having the divide between two different and somewhat intersecting UIs. Here’s the breakdown of changes in this new current redesign proposal:

• We started with a run down of what identity & authentication environments are most common in actual practice. Some of the combos you can select in the current UI (for example, hesiod and winbind) are not actually that common in practice, yet are just as easy to select in that UI as more common scenarios like NIS and kerberos. So now the proposed UI handles only the five most common environments and wacky combinations are not possible. (We figure, if you’re picky enough to want to do something that wacky, you’re comfortable enough setting it up yourself with either the command-line setup program or in the config files.) Through weeding out the less useful combinations, the UI provides a little bit more guidance to the user as to what they might want to select.
• The main focus is on the environments users need support for, not on the backend (legacy vs sssd) used to provide it. The ‘backend technology’ column of this table shows which backend is called into action for which UI choices. You’ll need the table to figure this out because it’s not embedded into the UI – and it shouldn’t be – do I really care whether or not the legacy stack or SSSD is letting me log into my company laptop? No, I just care that I can log in and get my work done.
• A group of us walked through the current advanced options tab and noted which options made sense in which scenarios, which didn’t, and figured out which really didn’t belong in the UI. We cut three options that made little practical sense and reworded another to make its utility to the user more clear.
• The current UI is based on a model of having a base dialog with multiple smaller dialogs that pop up on demand. Why so many windows? The new proposed UI mockups embed the form fields directly in the UI and different fields appear based on which dropdown entries you choose – no more baby windows popping up everywhere.
• Ray inspired the sanity-checking form field bits – I was worried about GDM error messages, but he noted it’s better to prevent user error upfront by sanity-checking user-entered fields – especially hostnames – to see if they even have a shot at working. The consequences here are pretty high – I can make a goof that prevents me from logging into my computer at all (unless I can break in via single-user mode.) The stakes are pretty high, so it’s a good idea to prevent error in the first place.

One interesting design problem that popped up was how to handle smartcards. Nalin and Ray very patiently explained to me how they work – the model is different than the other technologies which really conflicted with my own mental model of the situation. Smart cards can be used to authenticate local and network accounts. So if I have a smartcard with the account ‘duffy’ on it, and I have both a ‘duffy’ local login on my laptop as well as a ‘duffy’ login on a corporate LDAP server – when I plug in my smartcard, how does it know whether or not it should log into my local account residing on my laptop, or my corporate account residing on the corporate LDAP server? If I understand correctly, you can’t just configure smart cards to only work for one or the other without affecting other login methods too. So rather than making smart cards another authentication method for LDAP as originally proposed, smart cards are an option in the ‘advanced settings’ tab that you can turn on for both local & remote or off for both local & remote.

Anyway, enough jibber-jabber. If authconfig-gtk / system-config-authentication is a tool you’ve used or if you work in an environment that has centrally-managed logins, please let me know if these mockups make sense or where they might not work in your environment. Or, if you see stupid mistakes or flaws in some of the assumptions / methods I’ve described here, please feel free to holler. We’d love to hear your feedback! (and at the very least, I hope this description of some of the design process behind the revamp is interesting for you ) The full current UI revamp proposal is here.