Improvements to RTCPeerConnection’s spec compliance in Firefox 24

You may have noticed that Firefox’s RTCPeerConnection API (the WebRTC API that enables real-time audio/video calls) has changed in Firefox 24.  We made improvements to our spec compliance in Firefox 24 (largely by moving more of our code to use webidl directly).  Hopefully this makes using WebRTC across different browsers easier.  We’ll be making more improvements, both to our API and the implementation underlying it, in future Firefox releases.

The main benefit of the changes in Firefox 24 is that RTCPeerConnection is now an EventTarget, so you can register for notification-events rather than using callbacks.

The notifications themselves are also to spec now (RTCSignalingState, RTCIceGatheringState and RTCIceConnectionState), though the old ones will still work, with warnings (see below).

Also, RTCDataChannel, with its more declarative API, is updated to more closely match the spec. It too is an EventTarget.

Things to watch out for:
– failureCallbacks are no longer optional (as decreed by the spec).
– Inline JS {} in place of new mozRTCIceCandidate() or new mozRTCSessionDescription() no longer works.

PS: Due to Bug 903741, you may have to look in Firefox 24’s new Browser Console (Ctrl(Command)+Shift+J) to see all RTCPeerConnection API errors. This will be addressed in the next version (Firefox 25).

Co-authored with Jan-Ivar Bruaroey, who made all the code improvements discussed in this post.

WebRTC for desktop is now in Nightly…

WebRTC for desktop is now in Nightly and will be in the first version of Aurora when it updates to Firefox 18.  We support getUserMedia, PeerConnection, and DataChannels.  We have a basic UI for getUserMedia which we expect to be updating in the coming weeks.

The code is behind a pref for now, pending more testing.  To enable our WebRTC code in Firefox’s Nightly desktop build, browse to “about: config” and change the “media.peerconnection.enabled” preference to “true”.  That’s it.

Bug reports are appreciated.  Please file them at https://bugzilla.mozilla.org under “Product:Core”, “Component:WebRTC”.  Here is a shortcut to file a WebRTC bug.

Our initial WebRTC demo page is here, and we expect to be adding lots more demos to it in the coming weeks.

If I seem in any way less enthusiastic than usual, it is only because I am exhausted.  :-)  This represents tons of hours of work from so many people on the Firefox team (too many people to name — especially because I’m afraid I’d forget someone — but you know who you are!!!).  Thank you to everyone who helped us land this “747” on the flight deck.

I’ll will try to blog regularly on our progress as we work to make this a great product feature for Firefox.

Opus for audio compression – there IS a great, free alternative

There’s a new audio compression algorithm (codec) called Opus that has just been approved for publication by the IETF, and it’s awesome.  It matches or outperforms other codecs at the vast majority of bitrates and audio bandwidths (narrowband to fullband, 6-512 kb/s, mono, stereo, speech, music, fixed and variable bitrates, variable frame sizes) and can seamlessly switch between modes during a session or call.

Opus was designed to work well for interactive applications (like VoIP and videoconferencing) as well as for streaming applications.  What’s more, it can pretty much replace ALL proprietary codecs.  Right now, the minimum set of proprietary codecs you’d need to cover most of the application space Opus covers is AMR-NB, AMB-WB, AAC-LD, AAC-LC, and HE-AAC — and Opus beats them all.

Oh, and it’s completely free and open-source compatible.  Check out: https://hacks.mozilla.org/2012/07/firefox-beta-15-supports-the-new-opus-audio-format/ for a detailed explanation by Tim Terriberry, Jean-Marc Valin, and Ralph Giles – Opus authors and fellow Mozillians.

Opus files can play in Firefox Beta (Firefox 15) today, and Opus was just adopted on Monday as the mandatory-to-implement audio codec by the rtcweb working group at IETF (the IETF folks charged with figuring out how WebRTC, the new standard for real-time communications like VoIP and videoconferencing, should work).

Opus provides a clearly state-of-the-art audio codec that is compatible with open-source projects, and we hope that Opus gets used more and more as the word gets out that is freely available and awesome.  So, please help spread the word!

navigator.mozGetUserMedia() has just landed in Firefox desktop nightly builds!

I know a lot of Mozillians have been waiting for this: Firefox nightly (http://nightly.mozilla.org/) for Desktop now supports navigator.mozGetUserMedia().

mozGetUserMedia is a new DOM API that allows a JavaScript application to easily access a user’s camera and/or microphone.  It’s part of the greater WebRTC project which will enable real-time video/audio/data connections from within the browser.  The spec is still under development in the W3C, and so this API is currently a prefixed API in Firefox.  We’d love help from the community to try it out over the next few months and let us know how it’s working and what improvements you’d like to see.

mozGetUserMedia is still preffed off by default, so you will have to browse to about:config and set media.navigator.enabled to true (it’s a boolean).  We don’t yet have the UI for the feature (to query permission for camera and microphone access and to communicate camera/microphone status), and we won’t be enabling mozGetUserMedia by default until we have UI.  Also, because this feature is brand new, we’re still working out a lot of the kinks.  There are some known issues that we’re in the process of resolving:

  • We don’t yet support mixing both audio and video into a single stream yet (you can obtain them individually though as two different streams). (Bug 773649)
  • We’re seeing clock drift, so the video or audio streams may appear to start lagging after a while (Bug 771834)
  • On some systems, capturing audio doesn’t work a second time without restarting the browser (Bug 773646)

Note: we don’t currently support “stream_url=URL.createObjURL(stream); video.src=stream_url;”.  Instead we support “video.src=stream;”.   It’s under discussion in the W3C as to which one (or both) will be required.

To look at some sample code and try out a simple demo, browse to https://people.mozilla.com/~anarayanan/gum_test.html and look at the page source.

Please help us test and let us know what you think.  If you find bugs while using mozGetUserMedia, please report them at bugzilla.mozilla.org (Product: Core, Component:WebRTC).  If you want to give feedback on this feature or our implementation of it, please post to Mozilla’s dev.media newsgroup.
Thanks in advance for any help you can give us on this!

I’m so excited that WebRTC is finally happening!  getUserMedia is the first major piece of a project that has the power to revolutionize the web and impact how we live our daily lives. And I’m thrilled that Mozilla is helping to make that possible.

The first part of WebRTC has landed!

The first part of WebRTC, including the pieces needed to support full getUserMedia, landed (and stuck!) in mozilla-central last Thursday (6/21).  The feature is currently preffed off, but this was a critical piece to start getting WebRTC support into Mozilla.   Considering we were landing over 500,000 lines of code (yes, 500K — that’s not a typo), things have gone very smoothly.

We’re hoping to move quickly onto landing full getUserMedia for desktop;  there will be no UI for the feature initially, so it will be hidden behind a config option (about:config) to start.  But we hope to have UI for the feature by Firefox 17.  Similarly we hope to have Android support for this feature in the Firefox 17 time frame as well.

The rest of WebRTC (which is still being developed by the webrtc/rtcweb working groups in the W3C and IETF, respectively) will coming to mozilla-central in the next couple of months.  Randell Jesup, who is the technical lead and module owner for WebRTC at Mozilla, has been posting the project landing plans to dev.planning over the past few weeks.  He’ll continue to update plans there, and I’ll blog here to keep people up-to-speed as we try to bring WebRTC into Mozilla this summer.

Many, many thanks to the awesome Mozilla WebRTC team who have been making this spec and real-time communication within the Firefox browser a reality: Randell Jesup, Eric Rescorla (EKR), Anant Narayanan, Ethan Hugg, Suhas Nandaku, Enda Mannion, Tim Terriberry, and Ralph Giles!

getUserMedia still image capture for Android has landed

Android still image capture support based on getUserMedia has made it into Firefox 15!  In terms of functionality, this first piece (still image capture)of getUserMedia is the programmatic equivalent of the <input type=”picture”>.   However, it lets app developers play with this API so Mozilla can get early feedback (as the spec for getUserMedia is still being defined), and it sets up the infrastructure for full getUserMedia support (i.e. streaming video and audio from the camera and microphone), which Mozilla hopes to have fully supported in Android and Desktop by Firefox 17.

I want to thank all of the people who helped make this happen, especially Anant Narayanan (who is working with me as technical lead for the getUserMedia project at Mozilla), Doug Turner, Johnny Stenbeck, Fabrice Desré, and Jonas Sicking.

WebRTC demo: Integrating WebRTC with a Social API

Please check out Anant’s post:

WebRTC video call at IETF-83 showing Mozilla’s Social API

It shows a WebRTC video call between two users who are signed into a Social API using BrowserID (now called Persona).  It’s a wonderful demonstration of some of the cool things that can be done by combining WebRTC with some of the other emerging web technologies.  We showed this at IETF-83 in Paris.

Previous Older Entries

Follow

Get every new post delivered to your Inbox.