Project Ideas on Multimedia Networking

Post your project ideas on multimedia networking such as VoIP, P2P, video processing, audio echo cancellation, etc.

Platform for Multimedia Story Development

Create a web based platform that allows a story writer to easily create stories (fiction) using multimedia tools. Such immersive stories will use existing social network, telephone network and web sites to create supplementary data for the story in real-time while the story is being followed. A new programming language (perhaps XML-based) should allow defining the hyperlinks between different parts of the story along with temporal data, e.g., when a link becomes active, or a phone number is available to be called, or blog post is added to supplement the main story. Embedding existing graphics, text, video, sound, as well as real-time communication primitives should be enabled using drag-and-drop user interface.

Service to report potential SIP interop problems

Build a server/hosted service to which any SIP user agent or proxy can send SIP INVITE, MESSAGE, REGISTER or other requests, and it responds with the potential problems in the SIP request from compatibility and interoperability point of view. There are many little details that are implemented differently than what the specification requires. For example, some implementations choke if the SIP message contains line termination of LF instead of CRLF, space after semi-colon in parameters, missing new line after SDP especially in TCP mode, urn instead of SIP URI in request URI or To headers and so on. RFC 4475 defines various torture tests for SIP parsers and there are automatic verification tools for torture tests but are usually not used by many SIP implementers mostly because of the overhead of doing such tasks and the overhead of fixing the new found bugs. The specifications recommend that the sender usually send the outgoing message in canonical forms but be able to accept incoming messages with torture tests. The goal of this project is NOT to do torture tests, but to identify items in the SIP message which are not in canonical form and may cause interoperability issues with other vendors that are non-compliant. Even if your SIP implementation is compliant with the specification, you want to inter-operate with other vendors, and this project will help you identify potential problems in your SIP message formater (not the parser).

When the server receives a SIP request, it parses the request identifying all non-conventional and non-canonical syntax in the message, and responds back with an error response, preferably 400 Bad Request, with message body containing a list of potential problems as warning and errors. Unlike having a network sniffer application that uses pcap or parses some log data, this application doesn't require installing anything on the user's premises. The end user just points its user agent or server at our server to make a SIP request to our server/service, and gets a detailed list of potential interoperability problems in the response. If the end user has a trace of the message, she can use the standard network client, nc, on Unix machines to send the request, or our web interface and get the list of potential problems. The server should accept the SIP request on both TCP and UDP.

Add H.264 video in SIP-RTMP gateway

The existing SIP-RTMP gateway as part of the RTMPlite project enables interoperability between Web/Flash Player and standard SIP endpoints for VoIP such as using Speex and G.711. Traditionally the proprietary nature of Flash's camera captured video (using Sorenson codec) disallowed interoperability with standard SIP endpoint without a video transcoding, because existing SIP video endpoints support standards such as H.261, H.263, H.264. With Flash Player version 11 (beta) containing native H.264 encoding for camera captured video, it is now possible to enable video path translation in SIP-RTMP gateway software. Apparently Wowza and Flashphone already can do this. The task of this project is to enable video path translation between Flash Player 11+ and standards SIP video end points such as Bria and Ekiga that support H.264. Language: Python, C/C++ (understanding).

Completed

We have finished video translation in our siprtmp gateway. The translation works between Flash Player 11.2 (beta) and Bria 3 (SIP). It also works with Ekiga (SIP) with some workaround. Please see the Flash Player bug and issue 49 for details on what was blocking it for a long time.

A global SIP-based communication service

Many major vendors have large investment in SIP-based communication but have built closed walled gardens of communication systems that work only within their customer base using SIP. For example, a Facetime user cannot connect to other regular SIP network, or Comcast VoIP interconnection is completely hidden from end users. The existing open source software projects such as SER and pjsip are being used for building closed systems. Existing public SIP service such as sip2sip.info or iptel.org have a maintenance cost which may prevent them from going to a global scale. Lack of users signing up for these services is another reason. Another problem is that as a user if I sign-up for a SIP account on iptel.org my SIP address will be user@iptel.org, but I prefer to use my existing email of the form kundan10@gmail.com as my SIP identity.

The goal of this project is to build an open and global Internet communication "service" where anyone will be able to connect with any one else without any restriction using SIP and RTC-Web. The service will provide a DHT-backed name service with sample clients implementation for desktop, mobile and browser to connect with each other and to existing SIP networks.

It will follow a public open source service (POSS) model where the software as well as service is open. Anyone with an email address or phone number will be able to sign-up for an account. The servers will use database/DHT (not RFC 3263) for lookup hence can use any email addresses as a SIP address-of-record (AoR). The servers will be hosted using Amazon cloud, so that based on the demand more servers can be launched easily.

What is different compared to existing systems? It will be like Skype but based on SIP and Web-RTC standards. Anyone can connect to anyone else, but with open protocols. It will be completely open for developers, end users as well as VoIP providers. Anyone can fix a bug, and this gets updated in the live service after review/testing. Anyone can be part of the service by taking PSTN termination responsibility. It will use your existing email address as your SIP AoR. It will use your existing phone number to forward call to/from. It will integrate both SIP and RTC-web connectivity. It will have open API which developers can build applications and user interfaces for.

Online Academic Conference System

Create an online conference system that allows participants to attend and presenters to present conference sessions on Internet and web. The idea is to have an academic conference with technical program committee, papers, sessions, schedule, etc., but everything online. Attendees do not need to pay for travel. The registration fee can be much less. The system will bring together researchers as well as developers, to discuss things in real-time. The system will archive all presentations including audio and video.

This is a large project and requires much more mature system than what we have at vvowproject (see demo). In particular, it requires efficient one to many party video conferencing, screen sharing, ability to quickly create new rooms and meeting places for people to catch up during session breaks, reliable PSTN and mobile connectivity for audio, video and presentation for those who do not or can not use PC with browser, and above all robustness and scalability to large number of participants. The project should be built with not-for-profit and public open source service license, where people can contribute to code as well as pay for hosting of servers, organizing of conferences, etc., to be self sustainable.

Integrating WebRTC with SIP/RTP

Build an application that integrated Session Initiation Protocol/Real-time Transport Protocol (SIP/RTP)-based communication and Google WebRTC project. The goal is to allow interoperability between a SIP phone and WebRTC application. The gateway can be used as a back-end server or on the client host, e.g., as a plugin or Javascript application, if possible. The WebRTC project keeps the signaling part outside, hence SIP can be used as the signaling part directly in the browser, or in a gateway. As a side project, building a simple SIP user agent that uses WebRTC for media stack will be a good starting point.

Gateway between RTMP and WebRTC

Build a gateway to perform translation between Flash based audio and video using RTMP and Google WebRTC. You can use the existing rtmplite project to perform the RTMP side of the processing. The goal is to allow a Flash application to talk to a WebRTC application and vice-versa using this gateway. The user can run the gateway either in the back-end server or on the client host itself, e.g., as a separate application or a browser plugin. Both signaling and media needs to be translated.

Python API for webrtc

Create a Python wrapper API to expose the peer connection, video engine and voice engine related functions of the Google WebRTC project. The goal is to allow any Python application to use a general purpose audio and video engine such as to implement a VoIP phone or Internet radio terminal. The external module should be easy to use and preserve existing function names and concepts of WebRTC. For example, a simple multicast Internet audio tool can be created as follows

from webrtc import VoiceEngine
voe = VoiceEngine()
ch = Channel(voe)
ch.set_local_receiver(port=1000, multicast="224.1.2.3")
ch.set_send_destination(port=1000, ip="224.1.2.3")
ch.start_playout()
ch.start_receive()
ch.start_send()

White-labeled Flash-to-phone

An earlier project, SIP-RTMP gateway, has a Flash application, VideoPhone.swf, which implements the call control and user interface for a simple web-based video phone. This project should create a white-labeled web-based video phone without any user interface, so that the call control can be done using HTML and Javascript outside the Flash application, and the Flash application is used to just connect to the SIP-RTMP gateway and to handle media. The goal is to allow third-party to change the user interface and build their own call control logic in its web application.

Completed

This project is completed and closed. The flash-videoio project was used and modified to create a white-labeled Flash-to-phone application. You can find the details in How to do SIP-based VoIP call?

Digital Video Frame

Digital video frame is a hardware, similar to digital photo frame, which not only allows you to do slide shows of photos and videos, but also shows your contacts and allows you to do video conferencing with your contacts. Existing video conferencing software have a huge problem of configuration and setup. The goal of your project is to build video conferencing and presence software which is always on, and very very easy to use. Configuration, if any, will be done from web outside the video frame, e.g., setting your contact list or display name. Once the video frame software launches, it displays full screen view with your local mirrored camera view and your contact list containing photos of your contacts. After some time of inactivity, the playlist will start playing. You can configure the playlist from the web interface outside the video frame. The playlist contains photos as well as videos, either statically configured or uploaded, or randomly picked from other sites like Flickr and YouTube. In future this will be extended to support Internet TV or radio style broadcasting of audio and/or video.

Clicking on a contact list item automatically makes a video call to that user, or if you are already in a call, then invites that user in the existing call. When you receive an incoming invitation, the photo of the caller flashes for some time. Additionally a ringing sound is played if you are not already in a call. When you click on the flashing photo, the call is accepted, or if you are already in a call then the caller joins your call. In the idle state the system will automatically disable your camera after some time of inactivity. You can use the videocity project to get more ideas on how to layout the videos in two-party or multi-party calls. You can use Flash Player or AIR platform so that you don't have to worry about portable video, camera and sound components -- but using Flash platform is not a requirement. You can use any signaling protocol and web service to allow quick and easy video communication. The primary goal is ease of use. Language: any, prefer ActionScript, Python.

Shared Content Viewing

Implement a Flash-based shared content viewing software. A server co-ordinates content timeline and access among multiple participants. It should be possible to perform shared web browsing and shared video viewing using the software such as on YouTube. Try to use an existing protocol such as XMPP or HTTP between client and server. If multiple people are in the sharing context, everyone views the same web page. A mouse click or keyboard input on one persons browser is propagated to others, and same action is taken on all browsers. It should be possible to configure a moderator -- in public share everyone is a moderator, whereas in private share only the initiator of the share can promote others as moderator. A public share should be accessible by anyone whereas private share only on invitation. Language: Python, ActionScript, JavaScript, HTML

RESTful interface for SIP application service

Build a RESTful interface to a SIP application server so that one can access SIP services from web clients. For example, "/login" could represent a list of currently logged in users, and doing a "GET /login/{email}" gets the contact information of a user identified by that email, "PUT /login/{email}" does a registration with the SIP system, and "DELETE /login/{email} does un-registration. Similarly, "/user/{email}" can represent the user profile and "/user/{email}/messages" could represent user's voice/video messages. Anyone could do a "POST /user/{email}/messages" but only the owner could retrieve his messages using "GET /user/{email}/messages". Finally, "/call" could represent a list of currently active or scheduled conferences, where you can join a call using "POST /call/{call-id}" with your email and session information, disconnect from the call using "DELETE /call/{call-id}/{your-participant-id}". As a first step you will build simple SIP application server with user profile, registration, messages and conference functions. Then you will define and implement various RESTful APIs with authentication and security so that the server can also be accessed from a browser or web clients. Finally, you will use a SIP-RTMP gateway to allow a Flash client to join the SIP conference. You will need to extend the RESTful architecture to allow persistent client-server connection and callback event notifications from server-to-client.

I have more ideas on how the project can be done -- please contact me if interested. You can use the SIP implementation of the p2p-sip project, REST server tools of the restlite project and SIP-RTMP gateway of the siprtmp project. For additional motivation on this project please see my blog article on REST and SIP. Language: Python.

RTSP-RTMP translation

These are two competing streaming protocols on the Internet: RTSP and RTMP. Instead of having a gateway that translates between the two protocols, it might be better to build an integrated client or integrated server -- you can record using RTMP and view using Quicktime (RTSP), or you can use the same client to access real-time streams from RTMP or RTSP. Since RTMP incorporates RPC along with streaming control and media path, whereas RTSP is just streaming control, a complete translation of all the functions may not be feasible. Use: rtmplite, p2p-sip. Language: Python (server), ActionScript (client)

Video conferencing in LAN using Flash Player

The project builds in three stages:

(1) Build a video loopback for Flash Player for capturing camera and displaying local video in high quality using Flash Player. You can start with the existing software mirror code, change it to use compressed loopback of camera, and tune the quality and bandwidth of the camera to keep high quality within 100 Mbps bandwidth requirement. Language: ActionScript.

(2) Incorporate the client-server connection using the rtmplite software to host a Flash video server on local host. Your captured video will be sent to the server and your display will play the video stream received from the server. Language: ActionScript, Python

(3) Experiment with two users connected to the server. You just need to support one video room or conference, where everyone connected to the server are in the same conference. You may want to use the ContainerBox of the videocity project, or may display multiple video as you like. Language: ActionScript

siph323: SIP-H.323 signaling gateway

Implement open source SIP-H.323 interworking function [paper, internet-draft, slides] in Python. I have implemented a closed source siph323 software in C/C++ before. The goal of this new project is to first implement ASN.1 PER in Python, then add H.323 implementation, and finally the interworking function. This is a very involved project because of the complexity of H.323 and unavailability of any Python based H.323 library. I recommend doing it in a group or 2-3 students. Language: Python

sippeer: P2P telephony over SIP

Implement open source P2P-over-SIP approach [paper, detailed, slide] to P2P-SIP in Python. I have implemented a closed source sippeer software before in C++ with nice API. The goal of this new project is to use the programmable server approach to implement the pseudo-code presented in the paper and implement a P2P-over-SIP adaptor. Do not worry about NAT and firewall traversal initially. Language: Python

sipum: SIP and RTSP based voice/video messaging server

Implement open source voice/video messaging server using SIP and RTSP [paper, slides] in Python. I have implemented a closed source sipum project before in C++ with nice API. The goal of this new project is to first implement a simple RTSP client and server, then use the SIP and RTP modules from the p2p-sip project to implement sipum, and finally provide a web interface to access the voice/video messages. You do not need to record in specific formats like snd or mov but can just dump the RTP/RTCP packets in a file. You should allow adding support for new codecs or transcoders so that even if recording was done in G.711, you can listen to message in Speex, for example. Later you can integrate this with siprtmp to allow access from Flash applications as well as videocity. Language: Python

sipconf: SIP/RTP-based conferencing server

Implement an open source SIP/RTP-based multiparty multi-platform conferencing server [paper, slides] in Python. I have implemented it before as a closed source sipconf software with nice API in C++. The goal of this new project is to utilize the programmable server approach, keep the codecs and transcoders as pluggable, support video, and implement recording and playback of conferences. The project should use the SIP and RTP library from the 39 peers p2p-sip project. The next step will be to merge SIP and Flash conferencing using the siprtmp software. Language: Python

sipvxml: SIP-based VoiceXML browser

Implement open source SIP VoiceXML browser [paper, slides] in Python. I have implemented it before in C++ as sipvxml software using a nice API. Two problems with the previous implementation: it was closed source, and used VoiceXML version 1.0. This new project requires implementation of VoiceXML 2.1 in Python using the SIP libraries from the 39 peers p2p-sip project. You can use any available text-to-speech library or keep it open pluggable. The goal is to implement language feature and state machine for VoiceXML. Language: Python

P2P Audio Tool on Windows and/or Mac

Implement a audio tool on Windows and/or Mac OS X that intercepts audio captured and played in the device, and allows you to share it with others, your friends, contacts, etc. The idea is to do transparently implement audio conference and shared listening and chatting without being specific to a tool such as Skype or MSN. Thus, if you are listening to some music and share your speaker device with a friend, he will be able to listen to the same music. If you share your audio device with your friend, he will be able to talk to you using audio conferencing. Use RTP and SIP to support the distributed signaling and media transport. The tool should have option to allow music versus spoken audio, and graphical user interface to allow connecting devices among each other and with friends. Further extend it to allow sharing camera device as well as display device (i.e., part of your monitor display). Language: C/C++. Some background in audio will be helpful.

Video Room Firefox Extension

Implement a video room extension to Firefox using exiting video room services on the Internet. The video room will allow you to automatically video chat with other people browsing the same site or web page. For example, if two people are watching a particular news item and have this extension installed, they will be placed in a video room where they can video chat with each other. Further enhance the experience on other browsers (IE, Safari?), using existing protocols (SIP, Jabber) and using imported contact list from visited web site (e.g., when visiting Linked-In web site, you can automatically video chat with any linked-in profile or friend). Language: JavaScript, and others.

Flash to SIP calls

Using Adobe's Flash Player and Python SIP code, implement a PC-to-phone calling system which should allow both inbound and outbound calls between web browser and SIP. I have some ideas and preliminary code. The ActionScript library will allow any third-party to build user interface for the web-based soft-phone. Language: Python, ActionScript

Completed

The project is completed and closed. You can try out or see the documentation and source code at SIP-RTMP gateway project page.

Flash RTMP server in Python

My project named Flash RTMP server in Python hosted at Google Code can be used to receive connections and media from the Flash Player application.

P2P-SIP in Python

Implement the current version of IETF P2P-SIP WG internet-draft. You may reuse the core p2p, dht and rfc3261 modules of the 39 peers project. The goal is to implement the core functions that allows incorporating external DHT algorithm, authentication as well as transport. I have a few suggestions to simplify the draft and hence the implementation. Language: Python.