Serving audio and video content to mobile devices is very
important for many portals and content providers.
We can provide multimedia content in three formats:
For downloadable content, there are many formats and codecs that we
can use. Not all devices support all of them, so we should check the
documentation for our target devices or use WURFL properties to check for
support on the fly.
Video and audio files come with two technologies: a container format
and one or more codecs inside. The most compatible container formats for
mobile devices are 3GP and 3GP2, created by the 3GPP organization (formed
with 3G companies). They are very similar to the MPEG-4 (MP4) format, so
many devices support both.
There are also devices with support for MPEG, Flash Video, AVI, Real
Audio, Real Video, MOV, and Windows Media Audio/Video containers. Most
devices support the H.263 and H.264 codecs.
In the audio world, the most standard formats today are MP3 and
MIDI, but some devices also support MP4, Real Audio, WAV, AAC, and other
audio formats.
1. Delivering Multimedia Content
If we want to deliver multimedia content, we need to first look at
the Accept header or WURFL properties
to determine whether the device supports the format we’re using. If so,
we can use the delivery methods defined earlier. Depending on the
device’s capabilities, it may download the entire file before playing it
or it may try to play it while downloading using HTTP streaming
techniques.
Warning:
Multimedia files are generally large. If we deliver
noncompatible files, the user will be paying for non-useful traffic
and will not be happy with us.
For the best HTTP streaming technique, we need the server to
support partial downloads. If your server doesn’t support it,
there is a great PHP script available at http://www.mobilexweb.com/go/phppartial.
The direct download technique works in almost every compatible
phone: Symbian, iPhone, Android, Nokia Series 40, etc.
2. Embedding Audio and Video
Not all devices support embedded multimedia in web pages.
On some devices, the display and CPU constraints make this feature
impossible.
2.1. Flash Video
Flash Lite (from version) 3.0 and Flash Player for mobile
devices support Flash Video format (FLV). For the compatible devices , like Symbian devices from
3rd edition FP1, you can embed any Flash
content, although it is better if you compile your SWFs for Flash
Player 8. In fact, the YouTube desktop version works well on Flash
Lite 3–enabled devices (it’s a bit slow on some devices, but it
works).
2.2. Object embedding
You can use an object tag to include
a video in a web page:
<object data="video.mp4" type="video/mp4" width="300" height="300" />
Safari will show an image with a play button. When the user
presses play, the video will open in a full-screen QuickTime Player
window instead of being embedded in the browser.
An alternative is the embed
tag, preferred for iOS 1 and 2.X:
<embed src="poster.jpg" href="video.m4v" type="video/x-m4v" />
The object tag also works on
Symbian devices.
2.3. HTML 5
Although at the time of this writing they are only
compatible with some mobile browsers, such as Safari on iOS 3.0 and
later, the audio and video tags defined in HTML 5 can also be
used to embed multimedia content. The behavior is similar to using the
object tag. The video element supports the usage of a
source child tag that allows us to
define different media files in different codecs and bit rates for
best compatibility. The normal syntax is:
<video src="url"
poster="some_optional_picture.png"
controls="true"
width="320"
height="240" />
Safari supports the formats 3GP, MOV, and MPEG-4, and the H.264
and AAC-LC codecs.
2.4. Reference movies for iPhone
Safari on iOS also supports “reference movies,” created
with QuickTime Pro or a similar tool. A reference movie provides a
list of movie URLs with different bit rates (for example, for WiFi,
3G, or EDGE), so QuickTime can select the correct one for the device.
In our embed or video tag, we point to this new file.
Note:
At http://www.mobilexweb.com/go/refmovie
you can find an Objective-C Mac tool provided by Apple to generate
iPhone reference movie files from the command line.
3. Streaming
Streaming audio or video is a difficult solution if we
want to be compatible with all devices. Different platforms support
different streaming technologies.
Some devices, including Symbian, Windows Mobile, and BlackBerry
devices, support the Real Time Streaming Protocol (RTSP). When a link with this
protocol is used (e.g., rtsp://server/content), the
default media player—Real Player, Windows Media, etc.—is opened. The
content can be a file to be streamed (a prerecorded audio or video file)
or a live event (radio or TV show, sports event, etc.).
Note:
RTSP is an open standard for establishing and controlling
sessions between two points. Most platforms use the Real-time
Transport Protocol (RTP) for media streaming (audio and video) and can
deliver either live events or on-demand multimedia content.
We should expect more Flash streaming services for mobile devices
when the full Flash Player is available for mobile devices. Today, Flash
Lite 3.0 devices should work with Adobe Flash Media Server or the Red5
open source alternative (http://www.osflash.org/red5).
For general audio and video streaming there are also other
streaming solutions, like the commercial Helix Media Delivery Platform
(http://www.realnetworks.com) and QuickTime
Streaming Server (http://www.apple.com/quicktime/streamingserver).
Apple also maintains an open source alternative called Darwin Streaming Server. (http://developer.apple.com/opensource/server/streaming).
A streaming server uses TCP or UDP but generally does not use
HTTP. Some proxies may have problems redirecting the server’s TCP or UDP
packages, so HTTP has some advantages. However, it also has more
overhead than TCP or UDP and it is not prepared for live streaming
events. Some mobile devices that use HTTP have to download the entire
file before playing it. Other devices will start playing the file while
downloading it.
3.1. HTTP Live Streaming
Apple has created a new way to deliver live streaming
using HTTP, called HTTP Live Streaming, which it has presented to the
IETF as a proposed Internet standard. It is supported by iOS 3.0 and
allows the transmission of live events using the same HTTP protocol we
know. In fact, this is the only streaming solution that works on the
iPhone.
Implementing HTTP Live Streaming requires some changes on the
web server end. The simplified explanation of the protocol is that on
the server, the live stream is buffered in little packages sent to the
client. It’s like transmitting a live radio show by sending a series
of 10-second MP3s.
Note:
The well-known Akamai application acceleration service
provider offers live streaming services for the iPhone from http://iphone.akamai.com. Influxis (http://www.influxis.com) also offers mobile streaming
services as a shared hosting solution, for the iPhone/iPad and
BlackBerry and Android devices.
HTTP Live Streaming supports the H.264 codec for video and AAC
or MP3 for live audio streaming, as well as a bandwidth switcher for
different qualities. However, the best feature is that it passes any
firewall or proxy because it is HTTP-based.
Note:
Carson McDonald has developed an open source Ruby and C
solution for doing HTTP Live Streaming from a server, and even using
Amazon S3 services. You can check out his blog at http://www.ioncannon.net
or read more about the project at http://www.mobilexweb.com/go/httplive.
Apple offers a prerelease toolkit for this solution called HTTP
Live Streaming Tools. You need to be part of the Apple Developer
Connection to download it. The open source IceCast Server also
supports iPhone streaming, from version 2.3.2. You can find more
information at http://www.icecast.org.
Note:
For Android, iPhone, and Flash Lite 3 or Flash Player 10
devices, we can safely upload video content to YouTube and embed it
in our websites. The devices will render this content properly using
the internal Flash Player or a native YouTube application.
