Windowing Systems

Windowing Systems, Basic Drawing, Events (Event Loop,

Animation, Double-Buffering)
Windows: C++ Examples • On Windows, I use virtual box and linux for Xlib
examples
– Install virtual box, download ubuntu (or another
linux distro) as an iso
– Open virtual box and create an ubuntu linux vm
– Select, but do not start, and update IDE
secondary master to point to linux iso
– Start VM and install should proceed
– Then
• Install g++ with
– sudo apt-get install g++
• Install X11 dev options with
– sudo apt-get install libx11-dev
Windows: VM Setup • I also map a local directory onto a subdirectory in my
home directory as follows:
– Install Guest Additions from the Devices menu of
the VM
– On your local machine create a folder
– From Machine > Settings > Shared Folders click
the add icon and create a new directory pointing to
that folder.
– Check the auto-mount and make permanent
options
– On your host OS, make sure that the folder
permissions make it accessible
– Create a symbolic link to /media/<shared folder> to
a subdirectory of your home directory.
Mac & Linux: C++ Examples • On Mac OS X, this is much simpler:
– Install a version of the X Window System
(Xquartz) from http://www.xquartz.org

• On Linux, this is trivial:

– You’re already running X. Just compile it.

• Samples posted on the course website, along

with a makefile (pay attention to the include for
the X11 libs).
Windowing Systems
Before Windowing Systems

6 CS349 | Multiple Windows

X Windows Design Criteria (~1986) • implementable on a variety of displays
• applications must be device independent
• must be network transparent
• support multiple, concurrent application displays
• support many different application and management
interfaces
• support output to overlapping windows
(… even when partially obscured)
• support a hierarchy of resizable windows
(… an application can use many windows at once)
• high-performance, high-quality text, 2-D graphics,
imaging
• system should be extensible

(from Scheifler & Gettys, 1986)

7 CS349 | Multiple Windows
 How to support multiple windows?

8
CS349 | Multiple Windows
How to support multiple windows?
Base Window System (BWS)

10 CS349 | Multiple Windows

Base Window System • Lowest level abstraction for windowing system
• Routines for creating, destroying, managing
windows
• Routes input to correct window
• Ensures only one application changing frame
buffer (video memory) at a time
– one reason why single-threaded / non-thread-
safe GUI architectures are popular

11 CS349 | Multiple Windows

Base Window System • Creates canvas abstraction for applications
– Applications shielded from details of frame
buffer, visibility of window, other application
windows
• Each window has its own coordinate system
– BWS transforms between
coordinate systems
– Each window does not
need to worry where it is
on screen, always assumes
its top-left is (0,0)
• Provides basic graphics routines
for drawing
12 CS349 | Multiple Windows
Window Manager • Provides conceptually different functionality
–Layered on top of Base Window System
–Provides interactive components for
windows (menus, close box, resize
capabilities)
–Creates the “look and feel” of each
window

13 CS349 | Multiple Windows

Window Manager • Application Window vs. Application
“Canvas”
–the window manager owns the window
(including its controls)
–the application owns the canvas

Owned by the
Owned by the window manager
application

14 CS349 | Multiple Windows

BWS vs. Window Managers • X separates Base Window System from Window
Manager
– Enables many alternative “look and feels” for
windowing system (e.g., KDE, GNOME,
fvwm…)
– One of the keys to its lasting power: Can
continue to grow by changing the Window
Manager layer
• BWS and Window Manager are separate
processes

15 CS349 | Multiple Windows

Motif (Stacking)

16 CS349 | Multiple Windows

DWM (tiling)

17 CS349 | Multiple Windows

KWin (compositing)

18 CS349 | Multiple Windows

BWS vs. Window Managers • OSX, Windows combine “BWS” and Window
Manager
• Trade-offs in approaches?
– Look and feel…
– Window management possibilities…
– Input possibilities…
• Conceptually, on both platforms, there is a
separation of canvas (assigned to application)
and window decoration/OS overlay handled by
window manager
– Lines do blur when combined, however
– e.g. Fast access menubar in window frame

19 CS349 | Multiple Windows

Basic GUI Drawing

20 CS 349 - X Windows Drawing

Local Coordinates • Any modern OS manages multiple windows
– where the window is located, whether it is
covered by another window, etc...
– enables drawing using local
coordinate system for window

(o, o)
(o, o) +x
+x

+y +y

21 CS 349 - X Windows Drawing

Drawing Models • Three different conceptual drawing models:

SetPixel(x, y, colour)
Pixel DrawImage(x, y, w, h, img)

DrawLine(x1, y1, x2, y2, colour)

Stroke
DrawRect(x, y, w, h, colour)

DrawLine(x1, y1, x2, y2, color, thick)

Region DrawRect(x, y, w, h, colour, thick, fill)
DrawText(“A”, x, y, colour)

22 CS 349 - X Windows Drawing

Drawing Options • Lots of options for drawing
– e.g. drawLine(x1,y1,x2,y2)
• what colour?
• how thick?
• dashed or solid?
• where are the end points?
• how should the ends
overlap?
– Observation: most choices are
the same for multiple calls to
drawLine
• How to communicate all the
options?
– lots of parameters?
– more functions that are more
specific?
– something else?

23 CS 349 - X Windows Drawing

Graphics Context • Gather all drawing options into a single structure and
pass it to the drawing routines
– In X, the GC structure
• All graphics environments use variation on this approach
– Java/C#: Graphics Object
– OpenGL: Attribute State
• In X, the graphics context is stored on server
– Switch between multiple saved contexts to reduce
network traffic
– But limited memory on server
– There is a default context
– The context is global to the application: need a policy
• Now we don’t separate client application and XServer UI
routines, but assumption of repeated attributes still
applies
– Drawing is interpreted in context of graphics object,
attribute state …

24 CS 349 - X Windows Drawing

Simplifying Drawing With Clipping • What are some other problems that might arise
when trying to draw on a computer display?

Clipping and the Painter’s Algorithm

 Clipping: More on this later …

26
CS 349 - X Windows Events
Code Demo: clipping.cpp • XSetClipMask
• XSetClipRectangles
if (!is_clipping)
XSetClipMask(display, gc, None);
else
XSetClipRectangles(
display, gc, 0, 0, &clip_rect, 1,Unsorted);

27 CS 349 - X Windows Events

Painter’s Algorithm
• The basic graphics primitives are primitive.
• To draw more complex shapes:
– Draw back-to-front, layering the image
– Called “Painter’s Algorithm”

28 CS 349 - X Windows Drawing

Painters Algorithm Analogy

fast and with music: http://youtu.be/ghHxTjXAnM4

29 CS 349 - X Windows Drawing

Implementing the Painters Alg • Keep an ordered list of “displayables”
– Order: Back – to – Front.
• Each displayable knows how to display itself on
the screen
– Base class with a “paint” method
– Derived classes for different kinds of
displayables: text, game sprites, etc.
• Repaint
– Clear window
– Repaint everything in the display list

30 CS 349 - X Windows Drawing

Displayable Base Class /*
* An abstract class representing displayable
* things.
*/
class Displayable
{
public:
virtual void paint(XInfo &xinfo) = 0;
};

31 CS 349 - X Windows Drawing

Displayable Text /* Display some text */
class Text : public Displayable
{ public:
virtual void paint(XInfo &xinfo)
{ XDrawImageString( xinfo.display,
xinfo.window, xinfo.gc, this->x,
this->y, this->s.c_str(),
this->s.length() );
}

// constructor
Text(int x, int y, string s):x(x), y(y), s(s) {}

private:
int x;
int y;
string s;
};

32 CS 349 - X Windows Drawing

Displayable Polyline /* A displayable polyline */
class Polyline : public Displayable {
public:
virtual void paint(XInfo& xinfo) {
XDrawLines(xinfo.display, xinfo.window, xinfo.gc, &points[0],
points.size(), CoordModeOrigin );
}

Polyline(int x, int y) { add_point(x,y);}

void add_point(int x, int y) {

XPoint p; // XPoint is a built in struct
p.x = x; p.y = y;
points.push_back(p);
}

private:
vector < XPoint > points; // XPoint is a built in struct
};
33 CS 349 - X Windows Drawing
 list<Displayable*> dList; // list of Displayables
Displaying the Display List
dList.push_front(new Background(…));
dList.push_front(new Paddle(…));
dList.push_front(new Ball(…));

/* Function to repaint a display list */

void repaint( list<Displayable*> dList, XInfo& xinfo) {
list<Displayable*>::const_iterator begin = dList.begin();
list<Displayable*>::const_iterator end = dList.end();

XClearWindow( xinfo.display, xinfo.window );

while( begin != end ) {
Displayable* d = *begin;
d->paint(xinfo);
begin++;
}
XFlush( xinfo.display );
}

34 CS 349 - X Windows Drawing

Events Demystified
Events and the Event Loop
Animation
Double Buffering

35 CS 349 - X Windows Events

Human vs. System

User Interactive System

perceive present

seconds milliseconds or
faster

express translate

36 CS 349 - X Windows Events

Events Defined 1. An observable occurrence, phenomenon, or an
extraordinary occurrence.
2. A message to notify an application that something
happened.
– Examples:
• Keyboard (key press, key release)
• Pointer Events (button press, button release,
motion)
• Window crossing (mouse enters, leaves)
• Input focus (gained, lost)
• Window events (exposure, destroy, minimize)
• Timer events (elapsed time)
• Sensor update
37 CS 349 - X Windows Events
Role of the GUI Toolkit, BWS, WM • Collect event information
• Put relevant information in a known structure
• Order the events by time
• Decide which application and window should get
the event
• Deliver the event
• Some events come from the user via the
underlying hardware; some from the window
manager. WinMgr App

Base Window System

Hardware

Keyboard mouse Display

38 CS 349 - X Windows Events
Role of the Programmer • Indicate what events you are interested in to BWS/WM or
within code
• Write code to:
– Receive and interpret that event
– Update program content due to event
– Redraw display to communicate to user what changed
• In modern languages (Java, C#, Javascript) the process
of registering for events and handling events is simplified
– Java – Listener model
– C# -- Delegate model
– Javascript – Looks like a Java/C# hybrid, but is not
• Apparently simpler, actually worse, rationale for not
being used. If you’re interested:
• http://www.quirksmode.org/js/introevents.html
– Of course the existence of three models means that event
handling doesn’t work the same way in all browsers.
Receiving Events • In X, programmer handled all event processing
• Applications get the next event using:
– XNextEvent(Display* display, XEvent* evt)
• Gets and removes the next event in the
queue.
• If empty, it blocks until another event arrives.

• Can avoid blocking by checking if events

available using:
– XPending(Display* display)
• Query number of events in queue, never
blocks.
40 CS 349 - X Windows Events
Selecting Events to “listen to”
• Don’t always need all of the events. (Why?)
// Tell the window manager which input events you want.
XSelectInput( xinfo.display, xinfo.window,
ButtonPressMask | KeyPressMask |
ExposureMask | ButtonMotionMask );

• Defined masks: NoEventMask, KeyPressMask, KeyReleaseMask,

ButtonPressMask, ButtonReleaseMask, EnterWindowMask,
LeaveWindowMask, PointerMotionMask, PointerMotionHintMask,
Button1MotionMask, Button2MotionMask, ..., ButtonMotionMask,
KeymapStateMask, ExposureMask, VisibilityChangeMask, ...

• See
– http://www.tronche.com/gui/x/xlib/events/types.html
– http://www.tronche.com/gui/x/xlib/events/mask.html

41 CS 349 - X Windows Events

Event Structure: Union • X uses a C union typedef union {
int type;
XKeyEvent xkey;
XButtonEvent xbutton;
XMotionEvent xmotion;
// etc. ...
}
• Each structure contains
at least the following

typedef struct {
int type;
unsigned long serial; // sequential #
Display* display; // display event was read from
Window window; // window which event is relative to
} X___Event

42 CS 349 - X Windows Events

Java Event Structure: Inheritance • Java (and C# and C++.NET and VB and
Objective C and Swift and …) uses an
inheritance hierarchy
• Each subclass contains additional information,
as required (not shown)

43 CS 349 - X Windows Events

 window = XCreateSimpleWindow(…);
eventloop.min.cpp XSelectInput(display, window, PointerMotionMask | KeyPressMask);
XMapRaised(display, window);
XFlush(display);

XEvent event; // save the event here

while( true ) { // loop until 'exit’ event
XNextEvent( display, &event ); // wait for next event
switch( event.type ) {
case MotionNotify: // mouse movement
cout << event.xmotion.x << "," << event.xmotion.y << endl;
break;
case KeyPress: // any keypress
exit(0);
break;
// Handle other kinds of events
}
repaint( ... ); // call my repaint function
}
XCloseDisplay(display);

44 CS 349 - X Windows Events

Code Review: eventloop.cpp • XSelectInput
• XNextEvent
• eventLoop
• KeyPress and XLookupString
– character vs. scan codes
• Resizing the window

45 CS 349 - X Windows Events

Java Event Handling • Java simplifies much of this
• Events are transmitted to onscreen objects
without programmer intervention
• To handle events, tell onscreen objects what
you’re interested in and what to do
• Simple version in code last day
Opening a Window in Java import javax.swing.*;

public class TestWindow extends JFrame{

public TestWindow(){
this.setTitle("My Window");
this.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
}

public static void main (String args[]){

TestWindow myWindow = new TestWindow();
myWindow.setSize(400, 300);
myWindow.setVisible(true);
}
}

47 CS 349 - Syllabus
X Windows: Advanced Techiques
Animation
Double Buffering
Clipping

48 CS 349 - X Windows Events

Animation • A simulation of movement created by displaying
a series of pictures, or frames.
• Goals:
– Move things around on the screen
– Repaint 24 - 60 times per second
(frames-per-second, frame rate, or “FPS”)
– Make sure events are handled on a timely
basis
– Don’t use more CPU than necessary
– Easily understood code

49 CS 349 - X Windows Events

Respond to Events (non-blocking)
}
while( true ) {
if (XPending(display) > 0) { // pending events?
XNextEvent(display, &event ); // yes, process them
switch( event.type ) {
// handle event cases here ...
}
}
handleAnimation(xinfo); // update animation objects
repaint(xinfo); // repaint the frame

This doesn’t block, but it doesn’t

work very well either…

50 CS 349 - X Windows Events

Key to animation: managing time
while( true ) {
if (XPending(display) > 0) { // pending events?
XNextEvent(display, &event ); // yes, process them
switch( event.type ) {
// handle event cases here ...
}
}
handleAnimation(xinfo); // update animation objects
repaint(xinfo); // repaint the frame

// sleep for about 1/30 second

const int FPS = 30;
usleep(1000000/FPS);
}
Much better, but
still not great…
51 CS 349 - X Windows Events
Screen Flicker During Drawing • Imagine you are animating two balls on the
display in a game of breakout
• As the balls get close to each other, you notice
that one seems to flicker
– Why?
• If you paint the balls sequentially, then you might
clear a rectangle of the first ball’s position, paint
the first ball in the new position, then clear rect
for the second ball and paint the second ball
• Ball 1 has a bite out of it
– Or may even vanish …
1
– Until next update 2
Double Buffering • Flickering: when an intermediate image is on
the display
– e.g.: Clear, then redraw strategies
• Solution:
– Create an off screen image buffer
– Draw to the buffer
– Copy the buffer to the screen
as quickly as possible
(hopefully between refreshes)
• In X manual
• In … MOST … other
situations, it is done for you
53 CS 349 - X Windows Events
Double Buffering // create off screen buffer
xinfo.pixmap = XCreatePixmap(xinfo.display,
xinfo.window,
width, height, depth); // size and *depth* of pixmap

// draw into the buffer

// note that a window and a pixmap are “drawables”
XFillRectangle(xinfo.display, xinfo.pixmap,
xinfo.gc[0], 0, 0, width, height);

// copy buffer to window

XCopyArea(xinfo.display, xinfo.pixmap, xinfo.window,
xinfo.gc[0],
0, 0, width, height, // pixmap region to copy
0, 0); // top left corner of pixmap in window

XFlush( xinfo.display );
54 CS 349 - X Windows Events
Java Code: set Double Buffering
• In java swing, double buffering is built into the
JComponent class
– More on this next day, but, basically, all UI
components in java swing inherit from
JComponent
– Two methods:
• public boolean isDoubleBuffered();
• public void setDoubleBuffered(boolean o);
– Enabled by default
– If set in top-level container, all
subcomponents will be double buffered
regardless of individual settings
• Why?
Understanding Modern GUI Toolkits
Widgets • GUI Toolkits contain sets of widgets that you can
use to build applications, and enable event
delivery to widgets
• So … what is a widget?
• A widget (also graphical control element or
control) is an element of interaction in a
graphical user interface (GUI), such as a button
or a scroll bar. Controls are software
components that a computer user interacts with
through direct manipulation to read or edit
information about an application. User interface
libraries contain a collection of widgets and the
logic to render these.
– Wikipedia (AKA the font of all knowledge)
Nesting Windows: Child vs. Sibling
Window
xInfo1.display = display;
xInfo1.screen = screen;
initX(argc, argv, xInfo1, DefaultRootWindow( display
),
100, 100, 800, 600);

xInfo2.display = display;
xInfo2.screen = screen;
initX(argc, argv, xInfo2, DefaultRootWindow( display
),
50, 50, 300, 200);
xInfo1.display = display;
xInfo1.screen = screen;
initX(argc, argv, xInfo1, DefaultRootWindow( display
),
100, 100, 800, 600);

xInfo2.display = display;
xInfo2.screen = screen;
initX(argc, argv, xInfo2, xInfo1.window, // Change
“root” window
50, 50, 300, 200);
58 CS349 | Multiple Windows
multiwindow.cpp: Sibling Window

59 CS349 | Multiple Windows

multiwindow.cpp: Child Window

60 CS349 | Multiple Windows

Widget • Child windows are essentially widgets
– Widget is a generic name for parts of an
interface that have their own behavior:
buttons, progress bars, sliders, drop-
down menus, spinners, file dialog
boxes, …
– widgets also called “components”,
“controls”
– Can have their own appearance
– Receive and interpret their own events
– Put into libraries (toolkits) for reuse

Widget from Wow Wow Wubbzy

61 CS349 | Multiple Windows
Java AWT, Java SWT, Java Swing • Each operating system implements a set of
native widgets
– What you would expect: buttons, checkboxes,
combo boxes, etc., etc.
– But each OS’s native widget set had
idiosyncrasies
• Java’s cross-platform goal required a decision:
– AWT, lowest common denominator
– Swing, roll your own for the JVM.
– SWT, platform specific
• Pluses and minuses to each approach.
– What are they?
Pop Quiz • Why Java?
• Why did we talk about XWindows at all?
Summary • Basic X architecture (client, server, network)
• Windows: opening, disposing
• Drawing
– Models (pixel, stroke, region)
– graphics contexts
– Painter’s Algorithm; Display lists
• Events (structure, selecting, event loop, etc)
• Animation
• Double Buffering
• Clipping

64 CS 349 - X Windows Events