Вы находитесь на странице: 1из 208

Flumotion manual

Murray Cumming, Flumotion Documentation Team <murrayc@openismus.com> Thomas Vander Stichele, Flumotion Documentation Team <thomas at fluendo dot com> Christian Fredrik Kalager Schaller, Flumotion Documentation Team <christian at fluendo dot com>

Flumotion manual

by Murray Cumming, Thomas Vander Stichele, and Christian Fredrik Kalager Schaller Copyright © 2004,2005,2006,2007,2008 Fluendo

Abstract

User manual for Flumotion.

Flumotion is a trademark of Fluendo.

manual for Flumotion. Flumotion is a trademark of Fluendo. Permission is granted to copy, distribute and/or

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License (GFDL), Version 1.1 or any later version published by the Free Software Foundation with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. You can find a copy of the GFDL at this link [ghelp:fdl] or in the file COPYING-DOCS distributed with this manual.

Many of the names used by companies to distinguish their products and services are claimed as trademarks. Where those names appear in any documentation, and the authors are made aware of those trademarks, then the names are in capital letters or initial capital letters.

Some versions of this document appear with navigation icons from the Tango project. These are licensed under the Creative Commons Attribution Share-Alike license.

DOCUMENT AND MODIFIED VERSIONS OF THE DOCUMENT ARE PROVIDED UNDER THE TERMS OF THE GNU FREE DOCUMENTATION LICENSE WITH THE FURTHER UNDERSTANDING THAT:

1. DOCUMENT IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES THAT THE DOCUMENT OR MODIFIED VERSION OF THE DOCUMENT IS FREE OF DEFECTS MERCHANTABLE, FIT FOR A PARTICULAR PURPOSE OR NON-INFRINGING. THE ENTIRE RISK AS TO THE QUALITY, ACCURACY, AND PERFORMANCE OF THE DOCUMENT OR MODIFIED VERSION OF THE DOCUMENT IS WITH YOU. SHOULD ANY DOCUMENT OR MODIFIED VERSION PROVE DEFECTIVE IN ANY RESPECT, YOU (NOT THE INITIAL WRITER, AUTHOR OR ANY CONTRIBUTOR) ASSUME THE COST OF ANY NECESSARY SERVICING, REPAIR OR CORRECTION. THIS DISCLAIMER OF WARRANTY CONSTITUTES AN ESSENTIAL PART OF THIS LICENSE. NO USE OF ANY DOCUMENT OR MODIFIED VERSION OF THE DOCUMENT IS AUTHORIZED HEREUNDER EXCEPT UNDER THIS DISCLAIMER; AND UNDER NO CIRCUMSTANCES AND UNDER NO LEGAL THEORY, WHETHER IN TORT (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE, SHALL THE AUTHOR, INITIAL WRITER, ANY CONTRIBUTOR, OR ANY DISTRIBUTOR OF THE DOCUMENT OR MODIFIED VERSION OF THE DOCUMENT, OR ANY SUPPLIER OF ANY OF SUCH PARTIES, BE LIABLE TO ANY PERSON FOR ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES OF ANY CHARACTER INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF GOODWILL, WORK STOPPAGE, COMPUTER FAILURE OR MALFUNCTION, OR ANY AND ALL OTHER DAMAGES OR LOSSES ARISING OUT OF OR RELATING TO USE OF THE DOCUMENT AND MODIFIED VERSIONS OF THE DOCUMENT, EVEN IF SUCH PARTY SHALL HAVE BEEN INFORMED OF THE POSSIBILITY OF SUCH DAMAGES.

Table of Contents

I. Using Flumotion

1

1. Introduction

4

Introduction To Streaming

4

Introduction To Flumotion

4

2. Quick Start

5

Installation

5

Starting Flumotion

6

3. Using The Assistant

16

Detailed Example

16

Summary

28

4. Architecture

30

Introduction

30

Components

30

Managers And Workers

32

5. Deployment

33

Service scripts For Startup

33

Configuration Files

34

System Users

44

Multiple Servers

45

6. Usage Scenarios

46

Live Streaming of a Conference

46

On Demand Streaming Of Files

50

7. Security

53

Authentication of Connections

53

Transport Protocol Between Processes

53

Firewall Issues

54

8. Optimization and Performance

56

Component Locations

56

Limiting Bandwidth

56

System Configuration

57

9. Debugging and Troubleshooting

60

Debugging Flumotion

60

Troubleshooting Flumotion

65

A. Flumotion Services

67

B. Installing Extra Dependencies

68

Installing Python Modules

68

Testing Python Modules

69

Installing GStreamer plug-ins

70

C. Under the hood

73

GStreamer

73

Twisted

73

D. Command-line Options

74

Starting the manager

74

Starting the worker

74

E. The Configuration Assistant

75

Flumotion Connections

75

The Welcome Page

81

The Scenario Page

82

The Production Page

83

The Test Video Producer Page

84

The Webcam Page

85

Flumotion manual

The TV Card Page

86

The Firewire Video Page

87

The Overlay Page

88

The Test Audio Producer Page

89

The Sound Card Page

90

The Firewire Audio Page

91

The Encoding Page

92

The Theora Encoder Page

93

The Dirac Encoder Page

94

The Smoke Encoder Page

95

The JPEG Encoder Page

96

The Vorbis Encoder Page

97

The Speex Encoder Page

98

The Mulaw Encoder Page

99

The Consumption Page

100

The HTTP Streaming Page

101

The HTTP Streamer (Audio and Video) Page

102

The HTTP Streamer (Audio Only) Page

103

The HTTP Streamer (Video Only) Page

104

The Disk (Audio and Video) Page

105

The Disk (Audio Only) Page

106

The Disk (Audio Only) Page

107

The Icecast Streamer (Audio and Video) Page

108

The Icecast Streamer (Audio Only) Page

109

The Icecast Streamer (Video Only) Page

110

The Content License Page

111

The On Demand Page

112

The Summary Page

113

The Administration Window

114

Glossary

116

II. Flumotion Components Reference

118

I. Producer Components

120

audiotest-producer

121

firewire-producer

122

icecast-producer

124

loop-producer

125

soundcard-producer

126

tvcard-producer

128

videotest-producer

130

webcam-producer

132

II. Converter Components

134

repeater

135

overlay-converter

136

III. Combiner Components

138

av-basic-watchdog-combiner

139

av-switch-combiner

141

single-switch-combiner

143

IV. Encoder Components

144

dirac-encoder

145

jpeg-encoder

146

mulaw-encoder

147

smoke-encoder

148

speex-encoder

149

theora-encoder

150

Flumotion manual

vorbis-encoder

152

V. Muxer Components

153

multipart-muxer

154

ogg-muxer

155

VI. Consumer Components

156

disk-consumer

157

http-streamer

159

http-server

163

shout2-consumer

166

VII. Bouncer Components

168

htpasswdcrypt-bouncer

169

ical-bouncer

170

ip-bouncer

171

VIII. Miscellaneous Components

172

porter

173

III. Flumotion Plugs Reference

175

IX. AdminAction Plugs

177

adminaction-loggerfile

178

X. Bouncer Plugs

179

bouncer-testtoken

180

bouncer-trivial

181

XI. Component Plugs

182

component-cortado

183

component-example

185

XII. IdentityProvider Plugs

186

identityprovider-example

187

XIII. Component Plugs

188

manager-example

189

manager-manhole

190

XIV. RateController Plugs

191

ratecontroller-fixed

192

XV. RequestLogger Plugs

193

requestlogger-file

194

XVI. StreamData Plugs

195

streamdataprovider-example

196

Bibliography

197

Index

198

List of Figures

2.1.

Starting the Configuration Assistant

7

2.2.

Starting a new Manager and Worker

8

2.3.

Connecting to the new Manager

9

2.4.

The Configuration Assistant's Welcome Page

10

2.5.

The Configuration Assistant's Scenario Page

11

2.6.

The Configuration Assistant's Production Page, Showing Test Inputs

12

2.7.

The Configuration Assistant's Summary Page

13

2.8.

The Administration Window

14

2.9.

Viewing the Streamed Content, Showing Test Inputs

15

3.1.

Starting a new Manager and Worker

17

3.2.

Connecting to the new Manager

18

3.3.

The Configuration Assistant's Welcome Page

19

3.4.

The Configuration Assistant's Scenario Page

20

3.5.

The Configuration Assistant's Production Page

21

3.6.

The Configuration Assistant's Webcam Page

22

3.7.

The Configuration Assistant's Overlay Page

23

3.8.

The Configuration Assistant's Sound Card Page

24

3.9.

Choosing Encoders

25

3.10. The Configuration Assistant's Summary Page

26

3.11. The Administration Window

27

3.12. Viewing the Streamed Content

28

4.1.

The Flumotion Architecture

30

4.2.

The Flumotion Architecture, with the Manager and Workers

32

5.1.

Service Configuration in Fedora Linux or Red hat Linux

33

E.1. The Connection Window

76

E.2. Connecting to a recently opened Manager

77

E.3. Starting a new Manager and Worker

78

E.4. Starting a new Manager and Worker

79

E.5. Connecting to the new Manager

80

E.6. Authentication for a running Manager

81

E.7. The Configuration Assistant's Welcome Page

82

E.8. The Configuration Assistant's Scenario Page

83

E.9. The Configuration Assistant's Production Page

84

E.10. The Configuration Assistant's Test Video Producer Page

85

E.11. The Configuration Assistant's Webcam Page

86

E.12. The Configuration Assistant's TV Card Page

87

E.13. The Configuration Assistant's Firewire Video Page

88

E.14. The Configuration Assistant's Overlay Page

89

E.15. The Configuration Assistant's Test Audio Producer Page

90

E.16. The Configuration Assistant's Sound Card Page

91

E.17. The Configuration Assistant's Firewire Audio Page

92

E.18. The Configuration Assistant's Encoding Page

93

E.19. The Configuration Assistant's Theora Encoder Page

94

E.20. The Configuration Assistant's Dirac Encoder Page

95

E.21. The Configuration Assistant's Smoke Encoder Page

96

E.22. The Configuration Assistant's JPEG Encoder Page

97

E.23. The Configuration Assistant's Vorbis Encoder Page

98

E.24. The Configuration Assistant's Speex Encoder Page

99

E.25. The Configuration Assistant's Mulaw Encoder Page

100

E.26. The Configuration Assistant's Consumption Page

101

E.27. The Configuration Assistant's HTTP Streaming Page

102

Flumotion manual

E.28. The Configuration Assistant's HTTP Streamer (Audio and Video) Page

103

E.29. The Configuration Assistant's HTTP Streamer (Audio Only) Page

104

E.30. The Configuration Assistant's HTTP Streamer (Video Only) Page

105

E.31. The Configuration Assistant's Disk (Audio and Video) Page

106

E.32. The Configuration Assistant's Disk (Audio Only) Page

107

E.33. The Configuration Assistant's Disk (Video Only) Page

108

E.34. The Configuration Assistant's Icecast Streamer (Audio and Video) Page

109

E.35. The Configuration Assistant's Icecast Streamer (Audio Only) Page

110

E.36. The Configuration Assistant's Icecast Streamer (Video Only) Page

111

E.37. The Configuration Assistant's Content License Page

112

E.38. The Configuration On Demand Page

113

E.39. The Configuration Assistant's Summary Page

114

E.40. The Administration Window

115

Part I. Using Flumotion

Table of Contents

1. Introduction

4

Introduction To Streaming

4

Introduction To Flumotion

4

2. Quick Start

5

Installation

5

Starting Flumotion

6

3. Using The Assistant

16

Detailed Example

16

Summary

28

4. Architecture

30

Introduction

30

Components

30

Managers And Workers

32

5. Deployment

33

Service scripts For Startup

33

Configuration Files

34

System Users

44

Multiple Servers

45

6. Usage Scenarios

46

Live Streaming of a Conference

46

On Demand Streaming Of Files

50

7. Security

53

Authentication of Connections

53

Transport Protocol Between Processes

53

Firewall Issues

54

8. Optimization and Performance

56

Component Locations

56

Limiting Bandwidth

56

System Configuration

57

9. Debugging and Troubleshooting

60

Debugging Flumotion

60

Troubleshooting Flumotion

65

A. Flumotion Services

67

B. Installing Extra Dependencies

68

Installing Python Modules

68

Testing Python Modules

69

Installing GStreamer plug-ins

70

C. Under the hood

73

GStreamer

73

Twisted

73

D. Command-line Options

74

Starting the manager

74

Starting the worker

74

E. The Configuration Assistant

75

Flumotion Connections

75

The Welcome Page

81

The Scenario Page

82

The Production Page

83

The Test Video Producer Page

84

The Webcam Page

85

The TV Card Page

86

Using Flumotion

The Firewire Video Page

87

The Overlay Page

88

The Test Audio Producer Page

89

The Sound Card Page

90

The Firewire Audio Page

91

The Encoding Page

92

The Theora Encoder Page

93

The Dirac Encoder Page

94

The Smoke Encoder Page

95

The JPEG Encoder Page

96

The Vorbis Encoder Page

97

The Speex Encoder Page

98

The Mulaw Encoder Page

99

The Consumption Page

100

The HTTP Streaming Page

101

The HTTP Streamer (Audio and Video) Page

102

The HTTP Streamer (Audio Only) Page

103

The HTTP Streamer (Video Only) Page

104

The Disk (Audio and Video) Page

105

The Disk (Audio Only) Page

106

The Disk (Audio Only) Page

107

The Icecast Streamer (Audio and Video) Page

108

The Icecast Streamer (Audio Only) Page

109

The Icecast Streamer (Video Only) Page

110

The Content License Page

111

The On Demand Page

112

The Summary Page

113

The Administration Window

114

Glossary

116

Chapter 1. Introduction

Introduction To Streaming

Streaming allows people to watch video or listen to audio without waiting for an entire file to download. For instance, the later parts of a video will download while you are watching the first moments, so your media player always has enough data to continue playing the video without interruption. This is a better experience for viewers and a better use of broadcasters' bandwidth and processing power.

Broadcasters typically wish to provide two forms of streaming content: Live streaming and on-demand streaming. For instance, a radio station might want to broadcast their audio live over the Internet, simultaneously as it is broadcast via FM radio. However, that same radio station might like to provide on- demand access to the archive of their recorded content.

Introduction To Flumotion

Flumotion can broadcast your content live or on-demand in a range of popular audio and video formats. It supports a wide range of input hardware thanks to its use of Linux, GStreamer and other open source software. You can get started quickly with Flumotion's simple user interface then just place a link to the streamed content on your website.

Flumotion allows processing to be spread across multiple machines, so your platform can scale to handle more viewers of more streams in more formats. Its open source architecture makes it more efficient and more flexible than competing systems, making better use of your hardware.

Flumotion can even stream Flash Video and Windows Media formats, which usually require their own specific server software. Capture from DVB-S or DVB-T inputs is also possible.

Optional components allow you to require authentication or payment before viewing, or add advertisements or logos into the streamed content.

Chapter 2. Quick Start

This chapter will introduce you to Flumotion by showing how you might set up a simple streaming system on your local machine. In later chapters we will explain how you can customize the configuration.

Installation

Each Linux distribution has a slightly different method for installing software. Here are the installation instructions for some common Linux distributions.

After installation, Flumotion should be available from the Application Sound & Video sub-menu on your panel. In addition, three binaries (flumotion-manager, flumotion-worker and flumotion-admin) should be available in your path.

Installation on Ubuntu Linux

To install Flumotion, open the Synaptic Package Manager from the System Administration sub-menu on the panel.

First make sure that the Universe repository is enabled, by selecting the Settings Repositories menu item.

Then find Flumotion in the package list, click the check box in the first column and select Mark for installation from the drop-down menu, and click Apply on the toolbar.

Alternatively, just do sudo apt-get install flumotion from the command line Terminal. sudo apt-get install flumotion from the command line Terminal.

Note

If you need a newer version of Flumotion that is not yet packaged officially by Ubuntu Linux, you might try using the Flumotion Ubuntu Linux repository, by adding this line to your /etc/apt/sources.list file, changing hardy appropriately if you are using a different version of Ubuntu Linux. /etc/apt/sources.list file, changing hardy appropriately if you are using a different version of Ubuntu Linux.

Note

deb http://www.flumotion.net/pkg/ubuntu hardy main

You can add this line in the Synaptic Package Manager by selecting the Settings Repositories menu item, selecting the Third-Party Software tab, and clicking the Add button.

Installation on Fedora Linux or Red Hat Linux

To install Flumotion, open Add/Remove Software from the System Administration sub-menu on the panel.

Then find Flumotion in the package list, in the Servers category, and click the Install button.

in the Servers category, and click the Install button. Note Alternatively, as the root user, just

Note

Alternatively, as the root user, just do yum install flumotion from the command line Terminal.

Quick Start

Quick Start Note If you need a newer version of Flumotion that is not yet packaged

Note

If you need a newer version of Flumotion that is not yet packaged officially by Fedora Linux, you might try using the Flumotion Fedora Linux repository, by following the Fedora download instructions at the gstreamer web site [http://gstreamer.freedesktop.org/download/ fedora5.html], which explain how to install new .repo files and import the GPG key.

Starting Flumotion

To start Flumotion, choose the Application Sound & Video Flumotion Streaming Server Administration menu item on your panel. This starts the Flumotion Connections Assistant.

A Simple Example

This example will show how to start a Flumotion streaming system with the simplest possible defaults, just to demonstrate the basics. You won't even need a camera or microphone because we will use some test inputs that provide computer-generated video and audio.

When the Flumotion Configuration Assistant first starts, you will see this dialog.

Quick Start

Figure 2.1. Starting the Configuration Assistant

Quick Start Figure 2.1. Starting the Configuration Assistant If you already had a Flumotion system running,

If you already had a Flumotion system running, then you could use this dialog to connect to it and administer it. But we will create a new Flumotion system by choosing the Start a new manager and connect to it option.

On the next page, click Forward to start and connect. We will learn more about Managers and Workers in the Architecture chapter.

Quick Start

Figure 2.2. Starting a new Manager and Worker

Quick Start Figure 2.2. Starting a new Manager and Worker After these processes have started, you

After these processes have started, you will see a confirmation page before actually connecting to the manager.

Quick Start

Figure 2.3. Connecting to the new Manager

Quick Start Figure 2.3. Connecting to the new Manager You will now see the detailed Configuration

You will now see the detailed Configuration Assistant.

Quick Start

Figure 2.4. The Configuration Assistant's Welcome Page

Figure 2.4. The Configuration Assistant's Welcome Page Click the Forward button to see the first real

Click the Forward button to see the first real page of the assistant, which allows you to choose a common scenario, such as Stream Live or Stream Files On Demand. We will choose the defaults Stream Live scenario so just click Forward to accept these defaults.

Quick Start

Figure 2.5. The Configuration Assistant's Scenario Page

Figure 2.5. The Configuration Assistant's Scenario Page The next page allows you to choose Production inputs.

The next page allows you to choose Production inputs. We will use the defaults - Test Video Producer and Test Audio Producer so just click Forward to accept these defaults.

Quick Start

Figure 2.6. The Configuration Assistant's Production Page, Showing Test Inputs

Assistant's Production Page, Showing Test Inputs We don't need to change anything on the remaining pages,

We don't need to change anything on the remaining pages, so you can now continue to click Forward until you reach the last page. You will pass pages for Test Video Producer settings, Overlay settings, Test Audio Producer settings, Encoding settings, Theora Encoder settings, Vorbis Encoder settings, Consumption settings (streaming or saving to disk), HTTP Streaming settings, HTTP Streaming (Audio & Video) settings, Content License settings, and then the Summary page.

Quick Start

Figure 2.7. The Configuration Assistant's Summary Page

Figure 2.7. The Configuration Assistant's Summary Page After you click Apply, the Flumotion Administration window

After you click Apply, the Flumotion Administration window will open, showing all the components in your chosen Flumotion system. The icon for each component will gradually change as each component is started. If any component has a problem then the icon will indicate this and you can click on the component to see details.

Quick Start

Figure 2.8. The Administration Window

Quick Start Figure 2.8. The Administration Window If you select the http-audio-video component you will see

If you select the http-audio-video component you will see a clickable URL button on the right- hand side, in the Statistics tab. Clicking this button should cause the appropriate client application to open and play the streaming video. For instance, the Totem Movie Player will open on most Linux systems.

If you have no suitable player application, you can instead click on the http-server-audio-video server component to see an alternative URL. This plays the streamed content via a client-side Cortado Java applet embedded in an otherwise static web page.

Quick Start

Figure 2.9. Viewing the Streamed Content, Showing Test Inputs

2.9. Viewing the Streamed Content, Showing Test Inputs When you have finished, stop all the components

When you have finished, stop all the components by selecting Manage Stop All from the menu, then close the window.

See the Using The Assistant chapter for more details about the Configuration Assistant. See also the Administration Window section.

Chapter 3. Using The Assistant

The Simple Example section briefly showed the Configuration Assistant. In this chapter we will look again in more detail, changing some settings to stream our own video and audio instead of the test inputs. This assumes that you have a working camera and a microphone attached to your computer or built in to your computer.

Flumotion uses XML configuration files that tell it what to do. This assistant helps to create a Flumotion configuration for simple scenarios such as live streaming of a single camera or on-demand streaming of pre- existing video files. The assistant is not sophisticated enough to create more complex configurations that are often needed in the real world. For instance, you might need to stream the same video in multiple codecs and at multiple resolutions. However, the assistant should help you to become familiar with Flumotion and can be useful later to generate a fragment of a configuration file.

Detailed Example

As in the Simple Example section, choose the Application Sound & Video Flumotion Streaming Server Administration menu item on your panel then choose the Start a new manager and connect to it option.

As the next page of the assistant explains, this starts a Flumotion Manager, and a Flumotion Worker which it manages. Clicking Forward will start these and connect to the Manager so we can specify what the worker should do. You will learn more about Flumotion Managers and Workers in the Architecture chapter.

Using The Assistant

Figure 3.1. Starting a new Manager and Worker

Assistant Figure 3.1. Starting a new Manager and Worker After these processes have started, you will

After these processes have started, you will see a confirmation page before actually connecting to the Manager.

Using The Assistant

Figure 3.2. Connecting to the new Manager

Using The Assistant Figure 3.2. Connecting to the new Manager You will now see the detailed

You will now see the detailed Configuration Assistant.

Using The Assistant

Figure 3.3. The Configuration Assistant's Welcome Page

Figure 3.3. The Configuration Assistant's Welcome Page Click the Forward button to see the first real

Click the Forward button to see the first real page of the assistant, which allows you to choose a common scenario, such as Stream Live or Stream Files On Demand. We will choose the defaults Stream Live scenario so just click Forward to accept these defaults.

Using The Assistant

Figure 3.4. The Configuration Assistant's Scenario Page

Figure 3.4. The Configuration Assistant's Scenario Page The next page allows you to choose Production inputs.

The next page allows you to choose Production inputs. Choose Web Camera for Video and Sound Card for Audio. If you don't have a web camera or a sound card then you can choose the test producer for either of these. Flumotion supports a wide range of audio and video production devices.

Using The Assistant

Figure 3.5. The Configuration Assistant's Production Page

3.5. The Configuration Assistant's Production Page Click the Forward button to see the Webcam page, on

Click the Forward button to see the Webcam page, on which you can specify which webcam should be used, with what resolution and frame rate.

The Size and Frame Rate drop-down controls will be filled with the choices supported by your camera. For this example, choose a size of 320 x 240 and a frame rate of 15fps or similar because the default settings would require too much processing power for the average desktop PC.

Using The Assistant

Figure 3.6. The Configuration Assistant's Webcam Page

Figure 3.6. The Configuration Assistant's Webcam Page In this screenshot you can also see the worker

In this screenshot you can also see the worker drop-down choice. This allows you to choose what computer this component should run on, for instance if the camera is attached to a different computer that is available via the network. This allows you to distribute tasks across several server computers. In this case we have just started a single local worker, so localhost is the only available choice. This assumes that the camera is attached to your current computer. See the Deployment chapter to learn how to create extra workers. The Configuration Assistant would then offer these workers in the drop-down choice after connecting to their manager.

Click the Forward button to see the Overlay page and just click Forward again to accept its default values. The overlay converter allows you to overlay some text on top of the video, such as a broadcaster's name. You may also display some images at the bottom of the video, such as a Creative Commons logo.

Using The Assistant

Figure 3.7. The Configuration Assistant's Overlay Page

Figure 3.7. The Configuration Assistant's Overlay Page On the following Sound Card page, you can configure

On the following Sound Card page, you can configure the sound inputs. Choose Microphone from the Input drop-down control. In most cases the other settings will already be correct.

Using The Assistant

Figure 3.8. The Configuration Assistant's Sound Card Page

3.8. The Configuration Assistant's Sound Card Page Click the Forward button to see the Encoding page.

Click the Forward button to see the Encoding page. Encoding compresses your stream to make better use of bandwidth than a raw stream and provides the content in a format expected by client players.

In the Basic version of the Flumotion Streaming Server, we offer a choice of high-quality free codecs. Most are part of the Ogg family of codecs provided by xiph. The full version also supports a wide range of licensed proprietary codecs such as Windows Media, MP3 or Adobe Flash, to support common media player software.

Using The Assistant

Figure 3.9. Choosing Encoders

Using The Assistant Figure 3.9. Choosing Encoders On this page, choose the default settings of Ogg,

On this page, choose the default settings of Ogg, Theora and Vorbis.

We'll use the default Ogg Theora format for Video and Ogg Vorbis for audio, so just click Forward to accept them.

We don't need to change anything on the remaining pages, so you can now continue to click Forward until you reach the last page. You will pass pages for Theora Encoder settings, Vorbis Encoder settings, Consumption settings (streaming or saving to disk), HTTP Streaming settings, HTTP Streaming (Audio & Video) settings, Content License settings, and then the Summary page.

Using The Assistant

Figure 3.10. The Configuration Assistant's Summary Page

Figure 3.10. The Configuration Assistant's Summary Page After you click Apply, the Flumotion Administration window

After you click Apply, the Flumotion Administration window will open, showing all the components in your chosen Flumotion system. The icon for each component will gradually change as each component is started. If any component has a problem then the icon will indicate this and you can click on the component to see details.

You will see producer components to take audio and video input from your microphone and camera; encoder components to create Ogg Vorbis and Ogg Theora data from these inputs; a muxer component to combine the encoded audio and video data into one synchronized stream; and a http-

audio-video streamer component (a http-streamer) to stream that content to viewers via HTTP on port

8880.

There is also a non-flow component, porter which simply listens on the port on behalf of the http- audio-video streamer component, allowing multiple streamer components to provide content to the same port, but via different URLs, perceived by viewers as different pages on the same server.

Using The Assistant

Figure 3.11. The Administration Window

Using The Assistant Figure 3.11. The Administration Window As described in the Simple Example section, you

As described in the Simple Example section, you can select the http-audio-video component to see a clickable URL button on the right-hand side, suitable for a streaming client application, or you can instead click on the http-server-audio-video server component to see an alternative URL with a Java applet.

Using The Assistant

Figure 3.12. Viewing the Streamed Content

Using The Assistant Figure 3.12. Viewing the Streamed Content When you have finished, stop all the

When you have finished, stop all the components by selecting Manage Stop All from the menu, then close the window. Alternatively you could just close the window to leave the streaming server running and later start Flumotion Streaming Server Administration again to reconnect to the manager, providing the default user/test username and password login.

Summary

The Flumotion Configuration Assistant is useful for a simple examples such as this, or to generate fragments of configuration files with the Export Configuration menu item. A real streaming system would often be more complicated and must be defined via XML configuration files. For instance, you may need to supply content for multiple client players (multiple codecs) and different bandwidths (different quality levels). You will also need to set up the Flumotion system as a service to run at start up. You can read more about these issues in the Deployment chapter.

Using The Assistant

The Flumotion Administration window also has an Import Configuration menu item. This is only useful for viewing the results of your edits to the XML files while creating a full Flumotion configuration. To administer a real running system you should instead just connect to a running Manager.

Chapter 4. Architecture

Introduction

A Flumotion system consists of several parts. This section introduces the relevant terminology and explains

how the parts work together. This will help you to Deploy your Flumotion system.

Components

A

single Flumotion system is called a Planet. It contains several components working together. Some

of

these are Feed components, for instance to receive data, encode it, and stream it. This group of Feed

components is called a Flow. You could imagine it as a flowing river on the surface of the Planet. Each Flow component outputs data that is taken as an input by the next component in the Flow, transforming the data step by step.

Other components may perform extra tasks, such as restricting access to certain users, or allowing users to pay for access to certain content (Bouncer components). This group of components is called an Atmosphere. Again, it is part of the Planet.

called an Atmosphere . Again, it is part of the Planet . Note Flumotion's planet metaphor

Note

Flumotion's planet metaphor does not go further than this. It only involves the flow and an atmosphere.

This diagram shows a Planet with Components in both the Atmosphere and Flow.

Figure 4.1. The Flumotion Architecture

and Flow . Figure 4.1. The Flumotion Architecture Feed Components Feed components are arranged in a

Feed Components

Feed components are arranged in a flow, connecting to each other in sequence. Each component may be

in only one flow and must be connected to another component in the same flow.

There are three types of feed components:

Architecture

Producer A producer only produces stream data, usually in a raw format, though sometimes it is already encoded. The stream data can be produced from an actual hardware device (webcam, FireWire camera, sound card, etc), by reading it from a file,

by generating it in software (test signals), or by importing external streams from

Flumotion servers or other servers.

A feed can be simple or aggregated. For example an aggregated feed might

produce both audio and video.

For example, an audio producer component (such as soundcard-producer) provides raw sound data from a microphone or other simple audio input. Likewise, a video producer (such as firewire-producer) provides raw video data from a camera.

Converter

A converter converts stream data. It can encode or decode a feed. It can combine feeds or feed components to make a new feed. It can change the feed by changing the content, overlaying images over video streams, compressing the sound, etc.

For example, an audio encoder component (such as the vorbis-encoder) can take raw sound data from an audio producer component (such as the soundcard-producer) and encode it, for instance as Ogg Vorbis or MP3. The video encoder component (such as the theora-encoder) can encode data from a video producer component (such as the firewire-producer), for instance as Ogg Theora or MPEG.

Note that combiner converters can take more than one feed. For instance, the single-switch-combiner component can take a master feed and a backup feed. If the master feed stops supplying data then it will output the backup feed instead. This could show a standard "Transmission Interrupted" page.

Muxers are a special type of combiner component. For instance, the ogg- muxer component combines audio and video (for instance, from vorbis- encoder and theora-encoder) to provide one stream of audiovisual data, with the sound synchronized correctly to the video.

Consumer

A consumer only consumes stream data. It might stream a feed to the network, making it available to the outside world, or it could capture a feed to disk.

For example, the http-streamer component can take encoded data (for instance, from a ogg-muxer component) and serve it via HTTP for viewers on the Internet.

Other consumers, such as the shout2-consumer component can even make Flumotion streams available to other streaming platforms, such as IceCast.

Other Components

The other components are part of the Atmosphere. They provide additional functionality to flows and the manager and are not directly involved in creation or processing of the data stream.

Here are some examples of other components:

Bouncer A bouncer implements an authentication mechanism. It receives authentication requests from a component or manager and verifies that the requested action is allowed. A bouncer could be implemented using a standard htpasswd file, an LDAP username/password backend, or any other mechanism.

Architecture

For instance, the htpasswdcrypt-bouncer), controlled by an Apache-style htpasswd format uese crypted passwords.

Managers And Workers

A Flumotion system consists of a few server processes (daemons) working together. A Worker process creates processes for the Components, and a Manager process tells the Worker what to do. The Flumotion administration user interface connects to the Manager, which in turn controls the Workers, telling it to start and stop components.

The components may be split across multiple machines, so there may be multiple Workers - usually one per machine. The entire system usually has only one Manager, on one of the machines. However, different arrangements, such as multiple workers and managers on one machine, are possible and may be useful in some circumstances in order to clearly separate different parts of your Flumotion system.

This diagram shows the Planet again, with specific components grouped into Workers on separate computers, and a Manager to manage the workers.

Figure 4.2. The Flumotion Architecture, with the Manager and Workers

The Flumotion Architecture, with the Manager and Workers After the manager process starts, it starts an

After the manager process starts, it starts an internal Bouncer component used to authenticate workers and components later on. Then it waits for incoming connections from workers. Then it can tell the workers to start their components. Those new components will also log in to the manager.

Note that the Streamer Worker in this example would typically be outside of a corporate firewall, because client users on the Internet must connect to it directly to receive content. See the Firewall section for more details.

Chapter 5. Deployment

This section describes how to setup and configure Flumotion for a real-world streaming media environment. See also the Security and Usage Scenario chapters.

Service scripts For Startup

You can start and stop Flumotion using the service script at /etc/init.d/flumotion. For instance, use /etc/init.d/flumotion start to start Flumotion and /etc/init.d/flumotion stop to stop it.

This script tells Flumotion to use the configuration files found in /etc/flumotion/ (see the Configuration Files section) and to write logs to /var/log/flumotion/.

Your Linux distro probably starts Flumotion using this script at startup time. If it does not then you may use your Linux distro's graphical control panel to specify that this should happen, as described here.

The Services control panel in Fedora Linux

On Fedora Linux or Red Hat Linux you may use the Service Configuration control panel to start and stop

the Flumotion service. Choose System+Administration Services from your panel. If you start Flumotion in this control panel then it will be started automatically at each system start.

Figure 5.1. Service Configuration in Fedora Linux or Red hat Linux

5.1. Service Configuration in Fedora Linux or Red hat Linux Specifying Service Startup Using the Command

Specifying Service Startup Using the Command Line

Fedora Linux

Fedora Linux provides the chkconfig command, which helps with the administration of the /etc/ rc*.d/ directories, which specify the services that should start at boot time.

Deployment

For instance, you can specify that Flumotion should start at boot time with this command, as root: /sbin/ chkconfig flumotion on

you can discover whether Flumotion is already specified to start at boot time with this command, as root:

/sbin/chkconfig --list flumotion

You can also use the service to start and stop Flumotion manually. For instance: /sbin/service flumotion start and /sbin/service flumotion stop

Generic Linux

On other Linux systems, such as Ubuntu Linux, you may edit the system's startup files directly. Assuming that you have a Flumotion service script at /etc/init.d/flumotion, you can cause this to be started when the system reaches a certain run level, by adding a symlink in the appropriate rc*.d/ directory. For instance,

cd

/etc/rc2.d

ln

-s /etc/init.d/flumotion S20flumotion

Configuration Files

The /etc/flumotion directory contains a managers directory and a workers directory, which should both contain XML files. These files may be in sub-directories and may be split up into separate XML files. See the Managers and Workers section of the Architecture chapter.

The Manager's Configuration

The managers directory typically contains a sub-directory for each manager (usually only one), containing a planet.xml file. This file contains an XML node for the planet. This in turn contains nodes for the Planet's manager, atmosphere and flow, which themselves contain component nodes.

and flow , which themselves contain component nodes. Note Alternatively, the manager , flow and atmosphere

Note

Alternatively, the manager, flow and atmosphere nodes may be in separate XML files, also with parent planet nodes. These files will then be merged at runtime by Flumotion, without changing the files. This can make the configuration clearer.

In the manager node, you can specify the manager's hostname and the port number and transport

protocol that it should use, but you can usually just use the defaults. You should always use the default

ssl transport protocol, for secure connections, unless you are running Flumotion on an embedded device

with very restricted resources.

By specifying localhost for the manager's hostname, you can also restrict access to workers that are running on the same computer, though this would rarely be appropriate for a real Flumotion system.

Components

Each component XML node specifies the component name and type, which worker process it should belong to, and what feeding component it should receive data from, along with the specific properties for the component type.

Deployment

See also the Components section of the Architecture chapter and the Components Reference section.

Eaters and Feeds

Each flow component, apart from producers (and apart from non-flow components such as bouncers), receives data from one or more other components. This data stream is called a Feed. The eater node specifies the name of the upstream component that feeds data to the component, in the child feed node. When the feeding component has more than one output (also known as a Feeder), the specific output must be named by placing it after the component name, separated by a colon. For instance, <eater><feed>firewire-producer:audio</feed></eater>.

Version

 

The component node's version attribute is optional. You might use this to specify the exact version of Flumotion that you are using, if you are using a component whose available properties are likely to change in a future version of Flumotion If you try to use the XML configuration file with another version of Flumotion then you will see a warning in the log files and in the administration UI.

Flumotion also uses this version number to help it adapt when using a newer version of Flumotion, choosing to interpret the configuration as it would have been interpreted by the earlier version of Flumotion. However, you should eventually update the configuration for the new properties supported by the component.

Project

 

The component node's project attribute is optional, defaulting to "flumotion". This specifies the set of components from which the component is available, with "flumotion" being the core set. To use a component from an additional set, you should specify the name of that set for the project attribute.

Additional components are available from the following sets:

flumotion-bouncer

Additional Bouncer components.

flumotion-flash

Components for Flash video streaming.

flumotion-

Components for WindowsMedia streaming.

windowsmedia

Plugs

 

Plugs allow extra features to be added to components. Each component may have certain sockets which allow the component to use appropriate plugs. Likewise, each plug is compatible with a particular socket.

For instance, see the Logging section, or the Limiting Bandwidth section.

Logging

Many components can output log files to disk. This is useful for troubleshooting, monitoring, or billing.

For instance, this example specifies an apachelogger plug for the Logger socket of a http-server component, and specifies the logfile property for that plug.

Deployment

<component name="http-server-one" type="http-server"

<plugs>

<plug socket="flumotion.component.plugs.loggers.Logger" type="requestlogger-file">

<property name="logfile">/tmp/flumotion-one.log</property> </plug>

</plugs>

</component>

Inspecting Components and Plugs

You can discover information about a component type, including the list of Feeders that it provides, and the properties that you can set, and the sockets that it has, by using the flumotion-inspect command. This command can also provide information about a plug.

For instance, flumotion-inspect firewire-producer.For instance:

$ flumotion-inspect firewire-producer

Component:

firewire-producer

Produce feeds from a Firewire/DV device.

Source:

flumotion.component.producers.firewire.firewire in flumotion/component/producers/firewire

Eaters:

(None)

Feeders:

audio

video

dv

Features:

admin/gtk: admin_gtk.py:GUIClass component: firewire.py:Firewire wizard: wizard_gtk.py:FireWireWizardPlugin

Properties:

framerate: type fraction, optional The framerate (in fps) guid: type string, optional The GUID of the device (a hexadecimal like 0xhhhhhhhhhhhhhhhh) height: type int, optional The height to scale to is-square: type bool, optional Whether to scale to a 1:1 pixel aspect ratio

Deployment

scaled-width: type int, optional The width to scale to before correcting width: type int, optional The final width, including correction

Clocking:

Needs synchronisation: True Clock priority: 160

Sockets:

Repeaters

A repeater component can avoid excessive use of network bandwidth. For instance, if both a http-

streamer and disk-consumer component on a remote computer should use the same ogg-muxer component as their feeding component, an intermediate repeater component on the remote machine

can be used as the feeding component instead. This means that the data is sent over the network only once,

to the repeater component.

The Manager Bouncer

The manager node contains just one special component, for the manager-bouncer, which controls access to the manager from the worker processes.

Because there is currently only one supported type of Manager Bouncer, the type attribute must be htpasswdcrypt-bouncer.

For the htpasswdcrypt-bouncer, the user name and crypted password can be provided via a property node, setting the data property. For instance, this example shows the default username of "user" with the default password of "test"

<component name="manager-bouncer" type="htpasswdcrypt-bouncer"> <property name="data">user:PSfNpHTkpTx1M</property> </component>

Alternatively, you may use an external file, by providing the file path as the value of a file property node. For instance,

<component name="manager-bouncer" type="htpasswdcrypt-bouncer"> <property name="file">/etc/flumotion/managers/default/htpasswd</property> </component>

See the htpasswdcrypt-bouncer reference section.

The workers must specify one of these user name and password combination when logging into the manager. See the Remote Workers Configuration section.

the manager . See the Remote Workers Configuration section. Note You can create the credentials, including

Note

You can create the credentials, including the crypted password, using the htpasswd command, as provided by the Apache web server package.

Deployment

For example, to create a file containing credentials for username “someone” and password “s3kr3t”, type this command in a terminal:

htpasswd -d -c passwords someone

The program will ask you to type the password twice:

New password:

Re-type new password:

Adding password for user someone

Check the generated file by typing cat passwords which will show: 1

someone:5jKUrPB0Xbzos

Note that, while it is possible to specify passwords on the command line when creating this file, by using the -p option, this is bad for security reasons. The command line is visible to anyone logged in to the machine, as well as recorded in the shell's history.

Other Bouncers

All Flumotion systems should have a Manager Bouncer, as described in the Manager Bouncer section. Some systems may require additional bouncers, for instance to restrict access to the streamer component to certain users or IP addresses. These bouncer components should be declared in the atmosphere node rather than the flow node.

For instance, you might add a htpasswdcrypt-bouncer component to the atmosphere like so:

<planet> <atmosphere> <component name="streamer-bouncer" project="flumotion" type="htpasswdcrypt-bouncer" version="0.5.2.1" worker="localhost"> <property name="data"><testuser:Z5WEs3ezyz0E2> </property> </component> </atmosphere>

You could then simply specify this bouncer as a property for the streamer components's node, causing the user's web browser or player to prompt the user for a username and password. For instance:

<component name="http-audio" project="flumotion" type="http-streamer" version="0.5.2.1" worker="localhost">

<property name="bouncer">streamer-bouncer</property">

See the Manager Bouncer section for instructions to create the htpasswd string. You may instead specify a filename as a property, allowing you to update the list of allowed users and passwords from a different system.

There is also an ip-bouncer Bouncer that can allow or deny ranges of IP addresses instead of demanding a password.

Deployment

Example

This example was exported from Flumotion's Configuration Assistant after choosing the test audio and video inputs. A node for the manager was then added.

<planet name="example-planet">

<manager name="example-planet-manager"> <!-- <host>something.somedomain</host> --> <!-- Uses the current hostname if not specified. -->

<!-- <port>7531</port> --> <!-- Uses 7531 for SSL or 8642 for TCP if not specified. -->

<!-- <transport>ssl</transport> --> <!-- Uses ssl if not specified. Otherewise tcp. -->

<!-- <certificate>default.pem</certificate> --> <!-- Uses /etc/flumotion/default.pem if not specified. -->

<component name="manager-bouncer" type="htpasswdcrypt-bouncer"> <property name="data">user:PSfNpHTkpTx1M</property> </component> </manager>

<atmosphere> <component name="http-server-audio-video" type="http-server" label="http-server-audio-video" worker="localhost" project="flumotion"

version="0.5.2.1">

<property name="porter-username">pycBUhaLQlwt</property> <property name="mount-point">/ogg-audio-video/cortado/</property> <property name="porter-password">nWbDPQzKKrMa</property> <property name="type">slave</property> <property name="porter-socket-path">flu-KbVvOp.socket</property> <property name="port">8800</property> <plugs> <plug socket="flumotion.component.plugs.base.ComponentPlug" type="component-cortado"> <property name="has-audio">True</property> <property name="buffer-size">40</property> <property

name="stream-url">http://localhost:8800/ogg-audio-video/</property>

<property name="framerate">5.0</property> <property name="height">240</property> <property

name="codebase">http://localhost:8800/ogg-audio-video/cortado/</property>

<property name="width">320</property> <property name="has-video">True</property>

Deployment

</plug>

</plugs>

</component>

<component name="porter-http" type="porter" label="porter-http" worker="localhost" project="flumotion"

version="0.5.2.1">

<property name="username">pycBUhaLQlwt</property> <property name="socket-path">flu-KbVvOp.socket</property> <property name="password">nWbDPQzKKrMa</property> <property name="port">8800</property> </component>

</atmosphere>

<flow name="default"> <component name="producer-audio" type="audiotest-producer" label="producer-audio" worker="localhost" project="flumotion"

version="0.5.2.1">

<property name="volume">1.0</property> <property name="rate">44100</property> <property name="frequency">440</property> <clock-master>true</clock-master> </component>

<component name="producer-video" type="videotest-producer" label="producer-video" worker="localhost" project="flumotion"

version="0.5.2.1">

<property name="pattern">0</property> <property name="framerate">50/10</property> <property name="width">320</property> <property name="height">240</property> <clock-master>false</clock-master> </component>

<component name="overlay-video" type="overlay-converter" label="overlay-video" worker="localhost" project="flumotion"

version="0.5.2.1">

<eater name="default"> <feed alias="default">producer-video:default</feed> </eater> <property name="show-text">True</property>

Deployment

<property name="xiph-logo">True</property> <property name="text">Flumotion</property> <property name="cc-logo">True</property> <property name="height">240</property> <property name="width">320</property> <property name="fluendo-logo">True</property> <clock-master>false</clock-master> </component>

<component name="encoder-video" type="theora-encoder" label="encoder-video" worker="localhost" project="flumotion"

version="0.5.2.1">

<eater name="default"> <feed alias="default">overlay-video:default</feed> </eater> <property name="keyframe-maxdistance">10</property> <property name="sharpness">0</property> <property name="quality">16</property> <property name="noise-sensitivity">1</property> <clock-master>false</clock-master> </component>

<component name="encoder-audio" type="vorbis-encoder" label="encoder-audio" worker="localhost" project="flumotion"

version="0.5.2.1">

<eater name="default"> <feed alias="default">producer-audio:default</feed> </eater> <property name="bitrate">64000</property> <clock-master>false</clock-master> </component>

<component name="muxer-audio-video" type="ogg-muxer" label="muxer-audio-video" worker="localhost" project="flumotion"

version="0.5.2.1">

<eater name="default"> <feed alias="default">encoder-audio:default</feed> <feed alias="default-bis">encoder-video:default</feed> </eater> <clock-master>false</clock-master> </component>

<component name="http-audio-video" type="http-streamer" label="http-audio-video"

Deployment

worker="localhost" project="flumotion"

version="0.5.2.1">

<eater name="default"> <feed alias="default">muxer-audio-video:default</feed> </eater> <property name="porter-username">pycBUhaLQlwt</property> <property name="mount-point">/ogg-audio-video/</property> <property name="porter-password">nWbDPQzKKrMa</property> <property name="type">slave</property> <property name="porter-socket-path">flu-KbVvOp.socket</property> <property name="burst-on-connect">False</property> <clock-master>false</clock-master> <plugs> </plugs> </component>

</flow>

</planet>

The Workers' Configuration

The workers directory typically contains an XML file for the worker that runs on the computer, containing an XML node for the worker. This in turn contains nodes for the manager which the worker should login to, and an authentication node which specifies how this login should be done. You can specify the manager's port and transport protocol, but you can usually just use the defaults.

protocol, but you can usually just use the defaults. Note The password is written as plaintext

Note

The password is written as plaintext in the worker's configuration file but, as long as your worker is using the SSL transport protocol it is not actually passed over the network as cleartext.

The actual list of components for the worker, and the properties for those components, is in the manager's configuration. The manager will tell the worker what components to start, with what properties, after the worker logs in to the manager. This keeps the majority of the configuration on one computer, where the manager runs.

The feederport node specifies an additional range of ports that the worker may use for unencrypted TCP connections, after a challenge/response authentication. For instance a component in the worker may need to communicate with components in other workers, to receive feed data from other components. In some cases the worker may open up UDP ports in this range as well, for instance if a component is a clock master, used for synchronization. Two ports are usually enough.

Example

This is an example of the configuration file for a worker.

<?xml version="1.0"?> <worker>

Deployment

<!--

You can override the name of the worker, which will typically be hostname:(xmlfilename) <worker name="default"> -->

<manager>

<!--

 

This specifies what manager to log in to. Compare with command-line options.

-->

 

<host>somehost.somedomain</host> <!-- <port>7531</port> --> <!-- Defaults to 7531 for SSL or 8642 for TCP if not specified. -->

<!-- <transport>ssl</transport> --> <!-- Defaults to ssl if not specified. -->

-

 

</manager>

<authentication type="plaintext">

<!--

 

This specifies what authentication to use to log in. Compare with command-line options.

-->

<username>user</username>

<password>test</password>

</authentication>

<feederports>8650-8651</feederports>

<!-- A small port range for the worker to use as it wants. -->

<debug>*:4</debug>

</worker>

Setting Up Remote Flumotion Workers

One of the most powerful features of Flumotion is its ability to distribute work over many separate computers. In this section, we cover basic configuration instructions for Flumotion when used in this way.

By default, the installation of Flumotion only allows connections to the manager from the local host, for security reasons. If you want to allow workers or administration clients to log in from other hosts then you should change the authentication settings and remove the host node from the manager's configuration file, usually at /etc/flumotion/managers/default/planet.xml.

You should then tell the Flumotion workers to log into the remote manager by adding the needed configuration in the workers's configuration file, usually at /etc/flumotion/workers/ default.xml. When configured for remote connections this file will look something like this:

<worker>

<manager>

<host>stream.test.com</host>

Deployment

<port>7531</port>

</manager>

<authentication type="plaintext"> <username>user</username> <password>test</password> </authentication> </worker>

In this example we have used stream.test.com as the example remote server where the manager is running and we are using the default Flumotion SSL port 7531.

and we are using the default Flumotion SSL port 7531. Note You could also connect the

Note

You could also connect the worker directly from the command line by using this syntax:

flumotion-worker -H stream.test.com -P 7531 -u user -p test.

You can get more information on these and other command line parameters by reading the flumotion-worker man page (type man flumotion-worker) or by running flumotion- worker -h on the command line.

System Users

You should have a dedicated linux user for the Flumotion services, usually called Flumotion. This user should be specified when you setup the startup scripts. Make sure that the user has access to the audio and video devices for sound and video capture, for instance by making the special Flumotion user a member of the appropriate groups.

Standard distribution packages, for instance with Ubuntu Linux or Fedora Linux, create this user with appropriate access to system devices.

create this user with appropriate access to system devices. Note Some distributions, such as Ubuntu Linux

Note

Some distributions, such as Ubuntu Linux and Debian Linux, make administration easier by having special audio, video, and plugdev groups with ownership of hardware devices. Any user which is a member of these groups may access these devices.

However, some distributions, such as Fedora Linux and Red Hat Linux, do not have a sound, video or media group policy. For these Linux distributions, you'll need to do some extra configuration.

You need to adjust permissions for these devices: /dev/snd* and /dev/mixer* for OSS (sound), /dev/video* for video devices and /dev/snd/* for ALSA (sound). Because Fedora Linux uses udev these changes will normally only last until you log out or shutdown, because Fedora only the current user has access to these devices by default.

To make these modified permissions permanent, you may create your own media group and change the security setup for your distribution to use that group. There are several places you need to change for this to work.

You could modify /etc/udev/permissions.d/50-udev.permissions and put your changes in /etc/udev/permissions.d/10-flumotion.permissions, which will then override the 50-udev.permissions.

Deployment

You should also look in /etc/security/console.perms to see what pam_console changes on login/logout.

Multiple Servers

Flumotion can distribute its work over several computers, making the best use of your hardware and network resources, and making it easier, for instance, to use multiple content inputs from different locations.

See the Remote Workers Configuration section to learn how to configure a distributed Flumotion system, and see the Security chapter to be aware of other issues when deploying a distributed Flumotion system.

The Flumotion Services company offers a remote management and support solution for these issues and can help integrate Flumotion with your existing systems.

Chapter 6. Usage Scenarios

This chapter describes some common uses of Flumotion, with suggested configurations and some advice about working with Flumotion in these situations.

Live Streaming of a Conference

This section describes how Flumotion may be used to provide live online video streaming of a conference, allowing talks to be seen even by people not physically attending the conference venue. In addition, this video content can be simultaneously recorded to disk for later viewing. The On Demand section describes how to stream previously-recorded content from disk.

End User Experience

Your aim in this situation is to provide a web address for each conference room, usually by providing links on your organization's website. You can tell people to navigate to this address in their web browser to view the content in the appropriate player application. You may also wish to provide alternative web addresses, for people who wish to view the content directly in a web page via the Cortado Java applet.

Hardware

For this example, we will assume that there are two conference rooms. Each conference room requires the following hardware:

1. A computer to encode the video. This must be attached to the network, with a fixed IP address and dedicated bandwith. It should have a FireWire input.

2. A video camera with a tripod. Ideally, the video camera will have a line-level audio input and FireWire output. Higher-quality cameras produce clearer images, with less noise, allowing the stream to be encoded more efficiently.

3. A microphone or access to the mixing desk for the room's own audio recording system. If you are using a mixing desk then you will need a sound cable to take its audio signal.

Naturally, you will need power sockets for all your hardware. For instance, at least one for the camera and one for the computer. Make sure that the power to the room is not turned off at unexpected times by the building management.

The audio input should be connected to the video camera, simplifying the synchronization of audio and video. The camera should then be connected to the computer via their FireWire ports.

be connected to the computer via their FireWire ports. Note A "line-level" [

Note

A

"line-level" [http://en.wikipedia.org/wiki/Line_level] audio input is a strong audio input,

as

opposed to a weaker microphone-level input. To know if your equipment provides a line-

level or microphone-level input you will need to be familiar with your audio hardware, such

as the microphone or mixing desk. You must also know whether your camera expects a line-

level or microphone-level input. You may need to configure a mixing desk appropriately, or use a pre-amplifier or attenuator, being careful to maintain the quality of the audio. Ideally, you can use the room's mixing desk and you have a camera that can take its line-level input directly.

Usage Scenarios

You should also have one central networked computer to aggregate the streams from the computers in each room. This computer will usually be the only one with access to the Internet, allowing it relay the streams to an external high bandwidth server which will actually serve the content to viewers. You may require two network cards in this computer, using one for the local network and one for the Internet

using one for the local network and one for the Internet Note You will need to

Note

You will need to talk to the venue's system administrator to arrange for a computer to bypass the firewall or connect through the firewall. The system administrator will usually also provide the static IP addresses for each computer.

Example Configuration

Remember that we have two conference rooms. We could create one Flumotion Planet configuration to handle both rooms, with workers for the computers in each room, and one manager on the central computer. But to simplify the configuration we will create two Flumotion Planets, meaning that we have two managers running on the central computer.

Workers

First we will configure the workers on the computers in each room. As explained in the Worker Configuration section, this doesn't require much XML , because we just tell the worker to log in to a manager which will tell it what components to start. Each conference room worker logs into a different manager, so it logs in to a separate port, and should have separate feederport ports. For instance:

<worker name="room1-worker"> <!-- We also have room2-worker and streamer-worker. -->

<!-- Tell the worker to log in to a manager to get instructions: --> <manager>

<host>10.0.0.120</host>

<!-- We have 2 managers running on this central computer. The one for room1 is on this port: -->

<port>7530</port>

</manager>

<authentication type="plaintext"> <username>flumotion</username>

<password>Pb75qla</password>

</authentication>

<feederports>8650-8651</feederports>

</worker>

For each conference room, we will also have one worker on the central computer, which will receive data from the worker in the conference room and stream it to the outside world. This can use much the same configuration file.

Usage Scenarios

Managers

For each room we will create a manager on the central computer, in a separate XML configuration file. This manager will accept logins from its workers on the same port as specified in the worker's configuration.

Each manager will tell its workers to create components. It will tell the conference room worker to create audio and video producer components and encoder and muxer components, as well as a disk- consumer component to save the archives for later user. It will tell the local worker to create a streaming component, receiving encoded data from the conference room worker.

receiving encoded data from the conference room worker . For instance: Note Instead of archiving the

For instance:

Note

Instead of archiving the content on each encoding computer, in each conference room, you could archive the content from all rooms centrally on one computer. However, there is always a risk of local network problems, so it is safer to archive the content where it is created.

<?xml version="1.0" ?> <planet>

<manager name="planet-room1"> <!-- no host means all interfaces <host></host -->

<port>7530</port>

<component name="manager-bouncer" type="htpasswdcrypt"> <property name="data">flumotion:dF4wh3SMa4q/2</property> </component> </manager>

<flow name="default">

<!-- Get the video from the firewire camera: --> <component name="video-producer" project="flumotion" type="firewire-producer" version="0.2.1" worker="room1-worker"> <!-- properties --> <property name="framerate">25/2</property> <property name="height">288</property> <property name="is_square">False</property> <property name="scaled_width">360</property> <property name="width">360</property> </component>

<!-- Add some logos and text to the video (from the firewire producer component) with an overlay: --> <component name="video-overlay" project="flumotion" type="overlay" version="0.2.1" worker="room1-worker"> <eater name="default"> <feed alias="default">video-producer:video</feed> </eater>

Usage Scenarios

<!-- properties --> <property name="cc_logo">True</property> <property name="fluendo_logo">True</property> <property name="height">288</property> <property name="show_text">True</property> <property name="text">Some Conference</property> <property name="width">360</property> <property name="xiph_logo">True</property> </component>

<!-- Encode the video (from the overlay component) as Ogg Theora: --> <component name="video-encoder" project="flumotion" type="theora-encoder" version="0.2.1" worker="room1-worker"> <eater name="default"> <feed alias="default">video-overlay</feed> </eater>

<!-- properties --> <property name="bitrate">360</property> </component>

<!-- Encode the audio (from the firewire producer component) as Ogg Theora: --> <component name="audio-encoder" project="flumotion" type="vorbis-encoder" version="0.2.0" worker="room1-worker"> <eater name="default"> <feed alias="default">video-producer:audio</feed> </eater>

<property name="bitrate">20480</property> <property name="channels">1</property> </component>

<!-- Combine the encoded video and audio: --> <component name="muxer-audio-video" project="flumotion" type="ogg-muxer" version="0.2.0" worker="room1-worker"> <eater name="default"> <feed alias="default1">video-encoder</feed> <feed alias="default2">audio-encoder</feed> </eater> </component>

<!-- Stream the content to the internet, from the central computer: --> <component name="http-audio-video" project="flumotion" type="http-streamer" version="0.2.0" worker="streamer-worker"> <eater name="default"> <feed alias="default">muxer-audio-video</feed> </eater>

<!-- properties --> <property name="bandwidth_limit">10</property> <property name="burst_on_connect">True</property> <property name="mount_point">/</property>

Usage Scenarios

<property name="port">8900</property> <property name="user_limit">1024</property> <property

name="logfile">/home/fluendo/conference/room1-worker.log</property>

</component>

<!-- Save the content to disk for later on-demand streaming: --> <component name="disk-audio-video" project="flumotion" type="disk-consumer" version="0.2.0" worker="room1-worker"> <eater name="default"> <feed alias="default">muxer-audio-video</feed> </eater>

<!-- properties --> <property

name="directory">/home/fluendo/conference/archive/room1-worker/</property>

<property name="rotateType">none</property> </component>

</flow>

</planet>

Suggestions

You may wish to limit the bandwidth used, as described in the Limiting Bandwidth section. You may also wish to create a log of all views of your on-demand content, for instance for billing purposes. See the Limiting Logging section.

To support a huge number of live viewers you will need to use multiple streaming servers, fed by a repeater component, with some load-balancing system to automatically direct users to an available streaming server. The Flumotion Services company offers a solution for this and can help integrate Flumotion with existing systems.

On Demand Streaming Of Files

This section describes how Flumotion may be used to provide on-demand streaming of archived audio or video files. Unlike live streaming, the user may fast-forward in an on-demand stream, or go directly to a certain point. Unlike a simple download, the user's client player can do this without downloading the content before that point. And unlike a simple download, playing can start before the entire file has downloaded, where the file format allows that.

file has downloaded, where the file format allows that. Note Not all file formats can be

Note

Not all file formats can be streamed on demand, because they require client players to download the entire file to obtain enough information for the client to start playing. For instance, files using the AVI container (usually with an .avi extension) cannot be streamed on demand.

Usage Scenarios

End-User Experience

Your aim in this situation is to provide a web address for each file, usually by providing links on your organization's website. You can tell people to navigate to this address in their web browser to view the content in the appropriate player application.

Hardware

For on-demand streaming you need only one server, with the audio or video files on that same server.

Preparation

You may wish to provide your content in a variety of streaming formats and several quality levels suitable for different client bandwidths. However, Flumotion does not yet automatically encode and cache on- demand content so you will need to create the different versions of your files manually.

On Linux there are several utilities to help with this. The Flumotion Services company also offer a service to do this.

For Flash Video files (.flv files), you must index the files in order to allow seeking, such as moving .flv files), you must index the files in order to allow seeking, such as moving forward or backwards in a file. On Linux, the flvtool2 tool can index these files. See flvtool2

Note

On-demand streaming requires your system to have accurate MIME type information. In the event of problems you might look at the /etc/mime.types file to check that the file extension is properly mapped to a correct MIME /etc/mime.types file to check that the file extension is properly mapped to a correct MIME type. See also the IANA official list of MIME types [http://www.iana.org/assignments/media-types/].

Note

Example Configuration

This example creates a http-server component, streaming files from a directory specified by its path property. You can view these files in a client player, by opening the URL, which is the hostname plus the mount point, plus the file name. For instance, http://localhost/on-demand/test.ogv.

For instance:

<planet name="admin">

<manager name="planet-room1"> <!-- no host means all interfaces <host></host -->

<port>7530</port>

<component name="manager-bouncer" type="htpasswdcrypt"> <property name="data">flumotion:dF4wh3SMa4q/2</property> </component> </manager>

Usage Scenarios

<atmosphere> <component name="http-server-ondemand" type="http-server" label="http-server-ondemand" worker="localhost" project="flumotion"

version="0.5.2.1">

<property name="mount-point">/</property> <property name="port">8800</property> <property name="path">/var/www/</property> <plugs> </plugs> </component> </atmosphere> </planet>

Suggestions

You may wish to limit the bandwidth used, as described in the Limiting Bandwidth section. You may also wish to create a log of all views of your on-demand content, for instance for billing purposes. See the Limiting Logging section.

To support a huge number of simultaneous viewers you will need to use multiple streaming servers, fed by a repeater component, with some load-balancing system to automatically direct users to an available streaming server. For ongoing systems, you will also need some way to automatically create the various versions of each file. The Flumotion Services company offers a solution for these issues and can help integrate Flumotion with existing systems.

Chapter 7. Security

This chapter explains some of the security concepts used by Flumotion and mentions details that a network administrator would want to know or specify, such as port numbers used by Flumotion and the types of encryption used by network connections. Flumotion uses a secure setup by default but its users should be conscious of the basics of security, and should read this chapter to know how to configure Flumotion for maximum security.

Remember, while it is possible to use Flumotion in a completely insecure mode, this would require you to actively set the configuration parameters to do so. For example, by default, Flumotion will use an SSL protocol so no communication is done in cleartext.

Authentication of Connections

In Flumotion, authentication is handled by bouncers components. For instance, the manager bouncer allows workers to log in to their managers and another type of bouncer allows viewers to see protected content via HTTP if a certain token parameter is set.

See also the Remote Workers Configuration section.

The Manager Bouncer

The bouncer that handles authentication to the manager is always started as part of the manager configuration. It is unlike any other component in the planet, since it's not started by a worker process, but integrated into the manager daemon itself.

Currently, Flumotion ships with only one supported manager bouncer type (htpasswdcrypt- bouncer), controlled by an Apache-style htpasswd format using crypted passwords. More types will be added in the future. See the Manager Bouncer Configuration section to learn how to specify allowed hosts, users and passwords.

Transport Protocol Between Processes

Managers, Workers and clients

Managers, workers and administration clients may communicate via one of two supported transport protocols: either TCP or SSL. By default Flumotion uses SSL, which encrypts the data so it is not easily readable by looking at the network traffic. You should use this default wherever possible.

The transport protocol can be specified in the manager's XML configuration file and must then be specified in each worker's XML configuration file. See the Deployment chapter for more details about this part of the configuration files.

for more details about this part of the configuration files. Note Technically speaking, TCP is the

Note

Technically speaking, TCP is the transport protocol in both cases. SSL is a way of encrypting the actual TCP packets. In the context of Flumotion, however, we refer to both as protocols, where TCP is the standard TCP protocol (with plaintext communication), and SSL uses Secure Socket Layer on top of TCP.

Security

TCP is the least secure because any communication using this transport protocol is readable as plaintext on the network. This means that anyone with access to the traffic on your network can see authentication information, remote method calls, and other information exchanged between the various processes.

other information exchanged between the various processes. Warning Because the TCP communication is visible as

Warning

Because the TCP communication is visible as plaintext to anyone on the network, you should not use this transport except for testing, or in cases where you secure the communication through some other means (for example, through an SSH channel).

Generating The Certificate

For SSL communication to work, the manager needs a PEM certificate file. On startup, the manager looks for flumotion/default.pem under the system configuration sysconfdir directory. You can specify a different file using the --certificate parameter when starting the manager.

When Flumotion is installed from a standard Linux distribution package, a default certificate file is generated for you, but at some point you may wish to regenerate this file. For instance, the Fedora Linux package currently just creates a dummy default.pem file using fake data, via openssl's make-dummy-cert program.

You may generate a real RSA private key and certificate using openssl. For instance:

openssl req -newkey rsa:1024 -keyout private-key.txt -nodes -x509 -days 365 -out certificate.txt

You will be asked to provide some details about yourself and your organization when running this command.

Then combine the private key and the certificate into one default.pem file. For instance,

cat private-key.txt &gt; ${target}

echo ""

cat certificate.txt &gt;&gt; default.pem rm -f private-key certificate.txt

&gt;&gt; default.pem

Components

Components communicate with each other via unencrypted TCP connections because encryption of audio and video data would have a large processor performance cost. You should consider this when designing your network. See the Firewall Issues section for more information.

Of course, when the components are on the same machine, a localhost network socket is used. See the Optimizing component locations chapter for more information about which components should not communicate across the network because they would require too much network bandwidth.

Firewall Issues

Security

Your network probably uses firewalls to protect computers on your local network from unauthorized access from outside your network. Network administrators should read this section to learn how to configure Flumotion to run on their network, or how to adapt their network for Flumotion

Port Numbers

As described in the See the Manager Bouncer Configuration section, the workers communicate with their manager on a certain port. By default this port is 7531 for SSL, though this can be changed in the configuration if necessary for your network, or if you are running multiple managers on one machine. See also the Transport Protocols section.

Each worker may also use some (usually a maximum of two) extra feederport ports to exchange actual unencrypted content data between components (after authentication). This range must be specified in the worker's configuration and each worker on the same machine must use a different range of feederport ports.

Clients (viewers or listeners of your content) will also connect to your streaming component (such as a http-streamer component) via a port such as 80, just as they would connect to a traditional HTTP web server such as Apache. Therefore, that worker (containing the streaming component) would typically be outside of your firewall, with the minimum necessary services running.

However, the streamer component must connect to an "upstream" feeding component (such as an encoder or muxer) on another machine. Note that, like all downstream components, the streamer connects to its feeding component, rather than the feeding component connecting to the streaming component. Therefore, the feeding component's port must be open on the firewall. Future versions of Flumotion might simplify firewall administration by allowing the feeding component to connect to the streaming component instead, avoiding the need to open the port in the firewall.

instead, avoiding the need to open the port in the firewall. Note You will probably want

Note

You will probably want to stream some content on port 80, because that is the default HTTP port used by web browsers, making the URL simpler. But, as with a traditional web server, port numbers under 1024 may only be used by a root process, and you should not run any part of Flumotion as root. But you may stream on another port, such as 8800 (via the porter component, and then redirect to port 80, for instance with iptables.

The porter component thus allows multiple streamer components to serve content on the same port, such as port 80, via different URLs.

For instance, here is an example entry for iptables:

# Add nat section for redirecting port 80 to 8800:

*nat -I PREROUTING -p tcp --dport 80 -j REDIRECT --to 8800 COMMIT

Chapter 8. Optimization and Performance

Component Locations

Feeding components provide data to other components. Some components, such as video producers, provide large amounts of raw data. You should avoid sending this raw data over the network where possible. For instance, try to place the encoder component on the same computer as the producer, so that only the smaller encoded data stream travels over the network. Of course, this is not always possible if you are encoding in multiple formats.

The components' properties can affect how much bandwidth they would need to send data across the network. A high bit rate, resolution, and frame rate will generally produce large amounts of data. For instance, a firewire-producer component will produce 245 MBit/sec if set to 720x568 pixels, 24 bits per pixel, and 25 frames per second (PAL). On the other hand, a firewire-producer component set to 384x288 resolution (Quarter PAL) will produce only 61 MBit/sec of data.

Higher resolutions and frame rates will also cause a higher processor load so you will need a fast machine to encode high quality streams. However, each component runs in parallel in a separate process, so Flumotion will make use of multiple CPUs and multiple CPU cores. However, the actual CPU load is dependent on the input signal, because more complicated content will require more processor time to encode. You should monitor the load on the CPUs, avoiding a CPU load of over 70%.

Limiting Bandwidth

The amount of data served by a streaming component can vary over time depending on the content, because more complicated content can not be compressed as much as simple content. It is therefore difficult to predict how much bandwith will be needed even when limiting the number of client users.

However, Flumotion can limit the total bandwidth of a component to prevent it from becoming excessive. This is a helpful guarantee in case your estimates for maximum bandwidth could be incorrect or if you wish to optimize for a more common case.

For instance, this example shows the use of the ratecontroller-fixed plug with a http-server component, to restrict its bandwidth to 200,000 bits per second.

<component name="http-server-one" type="http-server"

<plugs> <plug socket="flumotion.component.misc.httpserver.ratecontroller.RateController" type="ratecontroller-fixed"> <property name="rate">200000</property> </plug> </plugs>

</component>

Optimization and Performance

System Configuration

This section explains how to fine tune some Linux system configuration parameters for improved performance.

File Descriptors

The streaming components use one file descriptor for each active client connection, such as an individual viewing your video. Hence, the limit on the number of file descriptors per process also limits the number

of client connections per process.

A Linux system has a global limit on the number of open file descriptors which is defined in the kernel and

global system configuration. Linux systems also typically have per-process limits which can be different for particular users or groups of users. There is both a soft and hard limit. The hard limit specifies the upper limit to which a process can raise its soft limit. The current limit for your process can be discovered using the ulimit -n command in a terminal.

Typically, systems are limited to 1024 open file descriptors for each non-root process. Depending on the system you run on, this limit can be changed in various ways.

On most distributions of Linux, such as Fedora Linux, Red Hat Linux, and Ubuntu Linux, , this limit can be changed like so:

1. Edit /etc/security/limits.conf as root. At the end of the file, add a line that reads

flumotion

-

nofile

8192

Replace flumotion with the username (or group name, prefixed with @) that the server will run under.

Replace 8192 with the maximum number of open file descriptors you want to allow.

2. Verify that this works by doing the following:

On Fedora Linux or Red Hat Linux, as root:

su -s /bin/bash -c "ulimit -n" flumotion

On Ubuntu Linux:

sudo su -s /bin/bash -c "ulimit -n" flumotion

This will show the allowed number of open file descriptors:

8192

will show the allowed number of open file descriptors: 8192 Note Due to the way PAM

Note

Due to the way PAM 1 and sshd are set up by default on Fedora Linux and Red Hat Linux, you might not be able to log in to this account any more using ssh. So let's correct that:

1. Edit /etc/pam.d/sshd as root. Make sure this file contains a line that reads

Optimization and Performance

session

required

pam_limits.so

2. Edit /etc/ssh/sshd_config as root. Turn off privilege separation by adding the following line

UsePrivilegeSeparation no

3. Restart the ssh daemon:

[root@server root]# service sshd restart Stopping sshd:

[

OK

]

Starting sshd:

[

OK

]

[root@server root]#

4. Verify that it works by trying to log in through ssh as the user:

[flumotion@server flumotion]$ ssh localhost flumotion@localhost's password:

[flumotion@server flumotion]$ ulimit -n

8192

Memory limits

Like all software, Flumotion and its dependencies can never be completely free of all bugs. In extreme cases, GStreamer might cause Flumotion to fail by trying to allocate too much memory, for instance due to a demuxing error or a programming error.

If you are lucky, the number is sufficiently small and the buffer can be allocated from actual RAM. If you're slightly less lucky, the number is so big that the machine just doesn't have enough RAM and swap to allocate, and the program terminates. (6 hours of CD quality audio is more than 4 GB). If you're really unlucky, the number is huge, but in between your available RAM and your available RAM + swap. In that case, the allocation will succeed, and this will drive your machine into a huge swap storm as soon as the buffer gets written to.

The same file that controls limits on the number of file descriptors can also control the amount of memory a process of a user is allowed to take. The current virtual memory limit for your process can be inspected using ulimit -v in bash(1).

Most Linux distributions have this set to unlimited by default.

Linux systems using PAM

On systems that use PAM, this limit can be changed:

1. Edit /etc/security/limits.conf as root. At the end of the file, add a line that reads

flumotion

-

as

512000

Replace flumotion with the username (or group name, prefixed with @) that the server will run under.

Replace 512000 with the maximum number of KB that you want to allow.

Optimization and Performance

2. Verify that this works by doing the following as root:

On Fedora Linux or Red Hat Linux, as root:

su -s /bin/bash -c "ulimit -v" flumotion

On Ubuntu Linux:

sudo su -s /bin/bash -c "ulimit -v" flumotion

Which will show the number of KB that a process is allowed to allocate:

512000

Chapter 9. Debugging and Troubleshooting

This chapter explains various methods for debugging problems with Flumotion. Though we are always working to make the server more robust and improve the quality of the feedback through our setup and configuration tools, you might sometimes need to do some investigation to diagnose your problem.

Debugging Flumotion

Many things can go wrong when streaming. Flumotion depends on a large stack of underlying technologies to function perfectly for your configuration. There is a huge array of hardware devices, drivers and libraries out there. So when something goes wrong it can be caused by anything from unsupported hardware, broken hardware, incomplete drivers to untested library versions. Fluendo and the Flumotion community are only able to test a small subset of the options available out there, so if you are not running a tested configuration you might run into problems. This chapter should help you debug such problems, and help you resolve them.

Debug levels

Flumotion and GStreamer log events at five different levels of increasing verbosity.

• 0: No debug output at all.

• 1: Only errors (default)

• 2: Errors and warnings

• 3: Errors, warnings and info messages

• 4: Errors, warnings, info, and debugging messages

• 5: Everything

Logging of events is also done in different categories. This allows you to log only certain events at a higher debug level than others. The debug variable can be set to a comma-separated list of category:level pairs. For example, to set the logging to level 3 for all categories, but to 4 for the admin category, component, you could run a workerwith

FLU_DEBUG=*:3,admin:4 flumotion-worker -u user -p test > worker.log 2>&1

Debugging on the Flumotion level

When Flumotion fails to run correctly the first place to start looking is, of course, at Flumotion itself. Even if the failure is caused by something further down the stack, Flumotion's internal error messages might be able to tell you what the problem is.

The Flumotion log files are by default placed under the /var/log/flumotion directory. The manager log is by default called 'manager.default.log' and the workerlog is called 'worker.default.log'.

Debugging and Troubleshooting

Inside these log files you will see information on the workings of the Flumotion server, with the level of detail depending on the log level used. By default relatively little information is stored in these files so in many cases you will need to increase the logging level in order to get enough information to solve the problem.

There are two ways of increasing the amount of debug information in the log files. The simplest is adding <debug>5</debug> to the top of your default.xml file under /etc/flumotion/workers, just underneath the <worker> tag.

The other way is to set an environment variable, FLU_DEBUG, for your process. This is the equivalent of setting <debug>5</debug> in the worker default.xml file.

Debugging GStreamer problems

Sometimes Flumotion problems are caused by issues originating from the GStreamer multimedia framework that Flumotion is built upon. And while these issues might not be in GStreamer, but even further down the stack, like the DV camera interfacing library or the video4linux driver for your specific camera, debugging at the GStreamer level will often help you identify your problem.

Applications like Flumotion can make use of the extensive GStreamer debugging system to debug pipeline problems. Elements will write output to this system to log what they're doing. It's not used for error reporting, but it is very useful for tracking exactly what an element is doing, which can come in handy when debugging Flumotion issues.

The GST_DEBUG environment variable is used to turn on GStreamer debugging.

GST_DEBUG=*:5 flumotion-worker -u user -p test > worker.log 2>&1

Also here the * can be replaced by a component name to get only debugging information related to that component written. For example, oggmux:5 would turn on debugging for the Ogg muxer element.

If as a result of reading the generated log files, you think that the cause of your problems are GStreamer plugins or elements not working as expected, there is a nice command line tool in GStreamer called gst- launch which you can use to do some manual tests. For instance if you are unsure if the v4l plugin is providing Flumotion with the needed data flow from your TV card you can test it by running a command like this:

gst-launch v4lsrc device=/dev/video0 ! ffmpegcolordec ! ximagesink

This pipeline tries to access the video0 Linux device and run the output through a color space converter then output it to the x window terminal. If this fails you know that either the GStreamer plugin or the v4l driver in question is broken somehow; the reported error might even tell you which.

Debugging other parts of the system

Sometimes neither Flumotion or GStreamer level debug output lets you understand why things are not working, although they should at least give you a better understanding of where in the stack the problem is to be found. We can not give you lot of general debugging advice for these underlying libraries except to suggest you look at their documentation and mailing lists for information. Sometimes even just searching for Google for information about your hardware device and Linux will turn up information about driver problems etc., with your specific hardware.

Debugging and Troubleshooting

Reading log files

Here are some tips to read Flumotion log files. First of all, make sure you are looking at a relevant part of the log file. The log files contain all the accumulated logging from multiple runs. Typically you are just interested in the current run you are analyzing. The manager, worker and job processes log four specific points in their lifetime at level 3 (INFO) that you can search for in the logs:

The process is starting. This gets logged as early as possible. If you want to check the logs for the last run, you go to the end of the log file and search backwards for this marker. Example:

INFO [17204]

manager

Mar 01 15:37:52

Starting manager 'planet' (flumotion/manager/main.py:253)

The process has started. From this point on, the process should keep running on its own; any case where it stops without being told to stop is a bug in the software and should be filed and fixed. Example:

INFO [17226]

worker

Mar 01 15:41:07

Started worker 'localhost' (flumotion/worker/main.py:315)

The process has been told to stop, and will now begin cleanup. For example, the workerwill wait for its child jobs to terminate.

INFO [17226]

worker

Mar 01 15:43:40

Stopping worker 'localhost' (flumotion/worker/main.py:329)

The process is ready to terminate. This is the last point it logs before exiting.

 

INFO [13329]

job

Mar 01 14:42:26

Stopped job '/default/producer-audio' (flumotion/job/main.py:96)

Look for important errors first. Start by searching for ERROR and WARN. Start with the first ones you see after the starting point.

Analyzing core dumps

Sometimes Flumotion will encounter a bug and core dump. This happens because of a bug in the lower layers - GStreamer, the Python interpreter, a support library,

It is important for the Flumotion developers to get a good stack trace from this core dump to be able to locate and fix the bug. To do this, you need to generate a stack trace from the core dump.

When this happens in a component, the UI will show a message like this one:

The component crashed. The core dump is '/var/cache/flumotion/core.21328' on the host running 'localhost'.

Debugging and Troubleshooting

Posted on Mon 24 Mar 2008 05:58:15 PM CET.

To produce a backtrace, you need to launch gdb on python with the given core file, like this:

# gdb python /var/cache/flumotion/core.21328 GNU gdb Red Hat Linux (6.3.0.0-1.122rh) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are

welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details.

This GDB was configured as "i386-redhat-linux-gnu" library "/lib/libthread_db.so.1".

Using

host libthread_db

Core was generated by `/usr/bin/python /usr/bin/flumotion-job /default/video-encoder-ogg /tmp/flumotion'. Program terminated with signal 11, Segmentation fault.

#0 0x4f40388a in _int_free (av=0xb3f00010, mem=0xb3f82230) at malloc.c:4410

4410 unlink(nextchunk, bck, fwd);

(gdb) thread apply all backtrace

Thread 5 (process 28840):

#0

0x008b2410 in ?? ()

 

#1

0xbfea3a38 in ?? ()

#2

0x00000008 in ?? ()

#3

0x00000004 in ?? ()

#4

0x4f45ecac in *

GI

poll

(fds=0x4f4ceff4, nfds=4, timeout=8)

at

/sysdeps/unix/sysv/linux/poll.c:87

#5 0x4f736363 in g_main_context_iterate (context=0xa0935d8, block=1, dispatch=1, self=0x9fc7120) at gmain.c:2849

#6

#7 0x006138ec in _wrap_g_main_loop_run (self=0xb7fd4a50) at pygmainloop.c:257 #8 0x4f587b72 in PyEval_EvalFrame (f=0xa0ea86c) at Python/ceval.c:3547 #9 0x4f588748 in PyEval_EvalCodeEx (co=0xb7d54760, globals=0xb7d4b3e4, locals=0x0, args=0xa1d588c, argcount=1, kws=0xa1d5890, kwcount=0, defs=0xb7c56c18, defcount=1, closure=0x0) at Python/ceval.c:2736 #10 0x4f586afb in PyEval_EvalFrame (f=0xa1d5714) at Python/ceval.c:3656 #11 0x4f588748 in PyEval_EvalCodeEx (co=0xb7b7f9a0, globals=0xb7b5af0c, locals=0x0, args=0xb7cd0638, argcount=1, kws=0xa1cb6a8, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2736 #12 0x4f53a5ae in function_call (func=0xb7affe64, arg=0xb7cd062c, kw=0xb7cfd13c) at Objects/funcobject.c:548 #13 0x4f522587 in PyObject_Call (func=0xb7affe64, arg=0xb7cd062c, kw=0xb7cfd13c) at Objects/abstract.c:1795 #14 0x4f585c38 in PyEval_EvalFrame (f=0x9f845e4) at Python/ceval.c:3840 #15 0x4f588748 in PyEval_EvalCodeEx (co=0xb7f5cda0, globals=0xb7f5b13c, locals=0x0, args=0x9fa08f0, argcount=2, kws=0x9fa08f8, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2736 #16 0x4f586afb in PyEval_EvalFrame (f=0x9fa074c) at Python/ceval.c:3656 #17 0x4f588748 in PyEval_EvalCodeEx (co=0xb7f5cde0, globals=0xb7f5b13c,

0x4f7366d9 in IA

g_main_loop_run

(loop=0xa127bd0) at gmain.c:2751

Debugging and Troubleshooting

locals=0x0, args=0x9f9d8ec, argcount=1, kws=0x9f9d8f0, kwcount=0, defs=0xb7f64718, defcount=3, closure=0x0) at Python/ceval.c:2736 #18 0x4f586afb in PyEval_EvalFrame (f=0x9f9d79c) at Python/ceval.c:3656 #19 0x4f588748 in PyEval_EvalCodeEx (co=0xb7f5ca20, globals=0xb7fb0824, locals=0xb7fb0824, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2736 #20 0x4f5887d3 in PyEval_EvalCode (co=0xb7f5ca20, globals=0xb7fb0824, locals=0xb7fb0824) at Python/ceval.c:484 #21 0x4f5a5538 in run_node (n=Variable "n" is not available. ) at Python/pythonrun.c:1265 #22 0x4f5a6c48 in PyRun_SimpleFileExFlags (fp=0x9f68008, filename=0xbfea5c4c "/usr/bin/flumotion-job", closeit=1, flags=0xbfea4914) at Python/pythonrun.c:860 #23 0x4f5a732a in PyRun_AnyFileExFlags (fp=0x9f68008, filename=0xbfea5c4c "/usr/bin/flumotion-job", closeit=1, flags=0xbfea4914) at Python/pythonrun.c:664 #24 0x4f5add55 in Py_Main (argc=3, argv=0xbfea49e4) at Modules/main.c:493 #25 0x080485b2 in main (argc=0, argv=0x0) at Modules/python.c:23

Thread 4 (process 28842):

#0

0x008b2410 in ?? ()

#1

0xb7a1f4e8 in ?? ()

#2

0x00000000 in ?? ()

Thread 3 (process 28843):

#0

0x008b2410 in ?? ()

 

#1

0xb6d11148 in ?? ()

#2

0xffffffff in ?? ()

#3

0x00000003 in ?? ()

#4

0x4f45ecac in *

GI

poll

(fds=0x4f4ceff4, nfds=3, timeout=-1)

at

/sysdeps/unix/sysv/linux/poll.c:87

#5 0x0030e4ef in gst_fdset_wait (set=0xa47bc00, timeout=-1) at gstfdset.c:523 #6 0x00312b29 in gst_multi_fd_sink_thread (sink=0xa4a8210) at gstmultifdsink.c:2269 #7 0x4f7508ef in g_thread_create_proxy (data=0xa470998) at gthread.c:582 #8 0x4f6e23b6 in start_thread (arg=0xb6d11ba0) at pthread_create.c:261 #9 0x4f46833e in ?? () from /lib/libc.so.6

Thread 2 (process 28846):

#0

0x008b2410 in ?? ()

#1

0xb4a4e438 in ?? ()

#2

0x00000003 in ?? ()

#3

0x00000000 in ?? ()

Thread 1 (process 14372):

#0 0x4f40388a in _int_free (av=0xb3f00010, mem=0xb3f82230) at malloc.c:4410

#1

#2 0x00817126 in theora_comment_query () from /usr/lib/libtheora.so.0 #3 0x00835910 in theora_decode_header () from /usr/lib/libtheora.so.0 #4 0x0082a724 in theora_clear () from /usr/lib/libtheora.so.0 #5 0x002f61aa in theora_enc_reset (enc=0xa4a6078) at theoraenc.c:314 #6 0x002f838b in theora_enc_chain (pad=0xa1f30e0, buffer=0xa41b510) at theoraenc.c:871 #7 0x41044ab9 in gst_pad_chain_unchecked (pad=0xa1f30e0, buffer=0xa41b510)

0x4f40741d in *

GI

libc_free

(mem=0xb3f82230) at malloc.c:3447

Debugging and Troubleshooting

at gstpad.c:3459 #8 0x410451ab in gst_pad_push (pad=0xa1f3020, buffer=0xa41b510) at gstpad.c:3625 #9 0x004f73c5 in gst_base_transform_chain (pad=0xa1f3560, buffer=0xa41b510) at gstbasetransform.c:1606 #10 0x41044ab9 in gst_pad_chain_unchecked (pad=0xa1f3560, buffer=0xa41b510) at gstpad.c:3459 #11 0x410451ab in gst_pad_push (pad=0xa1f34a0, buffer=0xa41b510) at gstpad.c:3625 #12 0x004f73c5 in gst_base_transform_chain (pad=0xa1f33e0, buffer=0xa41b510) at gstbasetransform.c:1606 #13 0x41044ab9 in gst_pad_chain_unchecked (pad=0xa1f33e0, buffer=0xa41b510) at gstpad.c:3459 #14 0x410451ab in gst_pad_push (pad=0xa1f3320, buffer=0xa41b510) at gstpad.c:3625 #15 0x00152c3a in gst_gdp_depay_chain (pad=0xa1f3260, buffer=0xa46f868) at gstgdpdepay.c:315 #16 0x41044ab9 in gst_pad_chain_unchecked (pad=0xa1f3260, buffer=0xa46f868) at gstpad.c:3459 #17 0x410451ab in gst_pad_push (pad=0xa1f31a0, buffer=0xa46f868) at gstpad.c:3625 #18 0x004f2996 in gst_base_src_loop (pad=0xa1f31a0) at gstbasesrc.c:1601 #19 0x4105dd66 in gst_task_func (task=0xb55d4598, tclass=0xa470698) at gsttask.c:192 #20 0x4f75236c in g_thread_pool_thread_proxy (data=0xa219c78) at gthreadpool.c:158 #21 0x4f7508ef in g_thread_create_proxy (data=0xa343328) at gthread.c:582 #22 0x4f6e23b6 in start_thread (arg=0xb544fba0) at pthread_create.c:261 #23 0x4f46833e in ?? () from /lib/libc.so.6 (gdb)

Create a bug report about the issue, including the output of gdb.

Troubleshooting Flumotion

This chapter will try to list some common issues you might encounter when trying to run Flumotion and provide solutions and workarounds for them.

Firewire Cameras

One common problem many people encounter is when trying to access their FireWire cameras. The problem is that Flumotion is not able to detect your FireWire device because the device creation for FireWire does not work on many current distributions. Check if you have a /dev/raw1394 device. If not create, it by using this command:

[root@server root]# mknod -m 666 /dev/raw1394 c 171 0

Flumotion worker refuse to start

A problem you might encounter after Flumotion has been stopped in an uncontrolled way, for instance due to a software crash, server failure, or you killing the Flumotion processes, is that the Flumotion

Debugging and Troubleshooting

workerrefuses to start. A common reason for this is that the pid file of the old process is still present. You need to manually remove it before you are allowed to restart the Flumotion worker. You can find the pid files in /var/run/flumotion. Make sure all Flumotion processes are stopped/removed before deleting the pid files manually.

device node /dev/raw1394 does not exist

Some udev based distributions like Fedora Core do not properly detect and create the needed device for Firewire cameras to work. This issue have been reported upstream and is being fixed. While waiting for the fix to be introduced you can solve it by running this command as root.

mknod -m 666 /dev/raw1394 c 171 0

GStreamer error: Could not open device xy for reading and writing

Some distributions, like Fedora Core, do not have permissions set up to allow Flumotion, when run under a separate user, to have read and write access to the needed devices like /dev/dsp*, /dev/mixer*, /dev/ video* and so on. You can change this by running chmod commands as root. For example:

chmod +rw /dev/dsp*

Appendix A. Flumotion Services S.A.

Flumotion Services S.A. is the company that develops and maintains the Flumotion software. They can help you deploy and administer Flumotion systems, providing additional Flumotion components and utilities where necessary, and integrating with your existing systems.

For large customers, their Flumotion Streaming Platform deals with issues encountered when streaming a large amount of content to a large number of users, controlling access to content and dealing with billing for the users. For instance, when streaming a popular sporting event.

The Flumotion Streaming Platform use clustering and load balancing software to distribute the load over several worker computers, allowing more simultaneous viewers than would be possible with a single streaming server.

The Flumotion Streaming Platform can also transcode content automatically, simplifying on-demand streaming when you wish to provide the content in many formats and many quality levels. Transcoding is also available as a separate service via FTP.

See flumotion-services

Appendix B. Installing Extra Dependencies

Installing Python Modules

Installing Python modules on Fedora Linux

On Fedora Linux, most Python modules are available in packages whose names start with python-. You can use yum to search for matching packages available to install. For example, if you want to install the dateutil module:

# yum search dateutil

Loaded plugins: refresh-packagekit ============================== Matched: dateutil =============================== python-dateutil.noarch : Powerful extensions to the standard datetime module

If it is available, you can install it (as root):

# yum install -y python-dateutil

Loaded plugins: refresh-packagekit fedora

| 2.4 kB

00:00

updates

| 2.3 kB

00:00

Setting up Install Process Parsing package install arguments Resolving Dependencies --> Running transaction check ---> Package python-dateutil.noarch 0:1.2-2.fc9 set to be updated --> Finished Dependency Resolution

Dependencies Resolved

=============================================================================

Package

Arch

Version

Repository

Size

=============================================================================

Installing:

python-dateutil

noarch

1.2-2.fc9

fedora

165 k

Transaction Summary

=============================================================================

Install

1 Package(s)

Update

0 Package(s)

Remove

0 Package(s)

Total download size: 165 k Downloading Packages:

Installing Extra Dependencies

(1/1): python-dateutil-1.2-2.fc9.noarch.rpm Running rpm_check_debug Running Transaction Test Finished Transaction Test Transaction Test Succeeded Running Transaction

Installing

: python-dateutil

| 165 kB

00:00

[1/1]

Installed: python-dateutil.noarch 0:1.2-2.fc9 Complete!

Installing Python modules on Ubuntu Linux

On Ubuntu Linux, most Python modules are available in packages whose names start with python-. You can use the Synaptic Package Manager to search for matching packages available to install.

On the command-line you may also use apt and apt-search. For example, if you want to install the dateutil module:

$ apt-cache search dateutil

python-dateutil - powerful extensions to the standard datetime module

If it is available, you can install it:

$ sudo apt-get install python-dateutil

Reading package lists Building dependency tree Reading state information

Done

The following NEW packages will be installed:

python-dateutil 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 0B/211kB of archives.

After this operation, 422kB of additional disk space will be used. Selecting previously deselected package python-dateutil.

Done

(Reading database

366470 files and directories currently installed.)

Unpacking python-dateutil (from Setting up python-dateutil (1.3-1)

/python-dateutil_1.3-1_all.deb)

Testing Python Modules

Once you have installed the package that provides the python module, you can test the installation by starting a Python interpreter and trying to import the module by hand.

If the module is properly installed then you will be able to import it without any errors:

Installing Extra Dependencies

$ python

Python 2.5.1 (r251:54863, Jun 15 2008, 18:24:56) [GCC 4.3.0 20080428 (Red Hat 4.3.0-8)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import dateutil >>>

If the module is not properly installed, then you will see an error when trying to import it:

$ python

Python 2.5.1 (r251:54863, Jun 15 2008, 18:24:56) [GCC 4.3.0 20080428 (Red Hat 4.3.0-8)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import dateutil Traceback (most recent call last):

File "<stdin>", line 1, in <module> ImportError: No module named dateutil >>>

Installing GStreamer plug-ins

Installing GStreamer plugins on Fedora Linux

On Fedora Linux, most gstreamer plugins are available in packages whose names start with gstreamer- plugins-. You can use yum to search for matching packages available to install. For example, if you want to install the schroedinger plugin:

# yum search schroedinger Loaded plugins: refresh-packagekit ============================ Matched: schroedinger ============================= gstreamer-plugins-schroedinger.i386 : GStreamer Plugins that implement Dirac : video encoding and decoding schroedinger.i386 : Portable libraries for the high quality Dirac video codec schroedinger-devel.i386 : Development files for schrodinger

If it is available, you can install it (as root):

# yum install -y gstreamer-plugins-schroedinger

Loaded plugins: refresh-packagekit Setting up Install Process Parsing package install arguments Resolving Dependencies --> Running transaction check ---> Package gstreamer-plugins-schroedinger.i386 0:1.0.3-2.fc9 set to be updated --> Processing Dependency: libschroedinger-1.0.so.0 for package:

gstreamer-plugins-schroedinger

Installing Extra Dependencies

--> Running transaction check ---> Package schroedinger.i386 0:1.0.3-2.fc9 set to be updated --> Finished Dependency Resolution

Dependencies Resolved

=============================================================================

Package

Arch

Version

Repository

Size

=============================================================================

Installing:

gstreamer-plugins-schroedinger i386 Installing for dependencies:

1.0.3-2.fc9

updates

40 k

schroedinger

i386

1.0.3-2.fc9

updates

211 k

Transaction Summary

=============================================================================

Install

2 Package(s)

Update

0 Package(s)

Remove

0 Package(s)

Total download size: 250 k Downloading Packages: