Flutter Video Chat App with WebRTC: Step-by-Step Guide

WebRTC − Real-time Communication for the Web

In the client-server model, when establishing communication between two clients, you are forced to put up with a delay that appears until the data frame from client 1 reaches the server and then client 2. To eliminate this delay, peer-to-peer (P2P) connections are suitable, where clients communicate (transfer data) to each other directly.

We are going to use this repository in our article. Don’t forget to leave a ⭐️ there.

GitHub - flutterwtf/Flutter-WebRTC

Contribute to flutterwtf/Flutter-WebRTC development by creating an account on GitHub.

GitHubflutterwtf

WebRTC is an open source project that allows you to directly exchange P2P without installing additional programs or plugins. Supported by all popular browsers today it is built on the basis of UDP. It makes no sense for us to delve into the stack, we are more interested in the process of installing and using such a connection.

To establish a P2P connection, we must know companion’s IP address so we can exchange data with him and he must know our IP. The STUN protocol (Session Traversal Utilities for NAT) will help us with this. We will not dwell on it in detail, but in short, STUN servers allow you to determine your public IP address and port, by which you can be reached from the external network.

In Flutter, we provide STUN servers when trying to create an RTCPeerConnection:

import 'package:flutter_webrtc/flutter_webrtc.dart';

static const Map<String, dynamic> _configuration = {
  'iceServers': [
    {
      'urls': [
        'stun:stun1.l.google.com:19302',
        'stun:stun2.l.google.com:19302',
      ],
    },
  ],
};

final peerConnection = await createPeerConnection(_configuration);

A connection cannot be established without the exchange of special configurations (RTCSessionDescription objects), so such an exchange is performed with a server help. In our case, for a quick start and simplicity, we will use Firebase tools.

More Detailed Explanation

(you can find code explanation below this table)

Handlers description:

All done, now both users know about each other and have fully configured RTCPeerConnection objects. When RTCPeerConnection receives MediaStreamTrack object onTrack handler adds this track to the existing MediaStream that, in it’s turn, could be used as a source for RTCVideoRenderer.

Implementing WebRTC in Your App

Permissions

Add the following lines to ios/Runner/Info.plist:

<key>NSCameraUsageDescription</key>
<string>$(PRODUCT_NAME) Camera Usage!</string>
<key>NSMicrophoneUsageDescription</key>
<string>$(PRODUCT_NAME) Microphone Usage!</string>

Then add features and permissions to android/app/src/main/AndroidManifest.xml:

<uses-feature android:name="android.hardware.camera" />
<uses-feature android:name="android.hardware.camera.autofocus" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.CHANGE_NETWORK_STATE" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />

RTCVideoRenderer Objects

Add flutter_webrtc as a dependency in your pubspec.yaml file.

Declare and initialize two objects, one for the local user and one for the remote one:

import 'package:flutter_webrtc/flutter_webrtc.dart';

final RTCVideoRenderer _localRenderer = RTCVideoRenderer();
final RTCVideoRenderer _remoteRenderer = RTCVideoRenderer();

@override
void initState() {
	_localRenderer.initialize();
	_remoteRenderer.initialize();
	super.initState();
}

@override
  void dispose() {
    _localRenderer.dispose();
    _remoteRenderer.dispose();
    super.dispose();
  }

The RTCVideoRenderer lets us play video frames obtained from the WebRTC video track. Depending on the video track source, it can play videos from a local peer or a remote one.

Turning on Camera and Micro

import 'package:flutter_webrtc/flutter_webrtc.dart';

Future<void> enableUserMediaStream() async {
  var stream = await navigator.mediaDevices.getUserMedia(
		{'video': true, 'audio': true},
	);
  emit(state.copyWith(localStream: stream));
}

Creating Room

Initialize RTCPeerConnection object (step 1 from table):

import 'package:flutter_webrtc/flutter_webrtc.dart';

Future<void> _createPeerConnection() async {
  final peerConnection = await createPeerConnection(_configuration);
  emit(state.copyWith(peerConnection: peerConnection));
}

Create offer from peerConnection object and send it to Firestore (step 2 from table):

import 'package:flutter_webrtc/flutter_webrtc.dart';

final offer = await state.peerConnection.createOffer();
final roomId = await _firebaseDataSource.createRoom(offer: offer);

...

import 'package:cloud_firestore/cloud_firestore.dart';
import 'package:flutter_webrtc/flutter_webrtc.dart';

class FirebaseDataSource {
  final FirebaseFirestore _db = FirebaseFirestore.instance;
  static const String _roomsCollection = 'rooms';

	Future<String> createRoom({required RTCSessionDescription offer}) async {
	    final roomRef = _db.collection(_roomsCollection).doc();
	    final roomWithOffer = <String, dynamic>{'offer': offer.toMap()};

	    await roomRef.set(roomWithOffer);
	    return roomRef.id;
	}
}

Add local stream tracks to peerConnection:

import 'package:flutter_webrtc/flutter_webrtc.dart';

state.localStream.getTracks().forEach((track) {
  state.peerConnection.addTrack(track, state.localStream);
});

Set event handlers (step 3 from table):

import 'package:flutter_webrtc/flutter_webrtc.dart';

void _registerPeerConnectionListeners(String roomId) {
  state.peerConnection.onIceCandidate = (candidate) {
    _firebaseDataSource.addCandidateToRoom(roomId: roomId, candidate: candidate);
  };

  state.peerConnection.onAddStream = (stream) {
    emit(state.copyWith(remoteStream: stream));
  };

  state.peerConnection.onTrack = (event) {
    event.streams[0].getTracks().forEach((track) => state.remoteStream.addTrack(track));
  };
}

...

import 'package:cloud_firestore/cloud_firestore.dart';
import 'package:flutter_webrtc/flutter_webrtc.dart';

class FirebaseDataSource {
  final FirebaseFirestore _db = FirebaseFirestore.instance;

  static const String _roomsCollection = 'rooms';
  static const String _candidatesCollection = 'candidates';
  static const String _candidateUidField = 'uid';

	// Current user id, used to identify candidates in Firestore collection
	// You can get it from FirebaseAuth or just generate a random string, it is used just
	// to understand if a candidate belongs to another user or not
  String? userId;

  Future<void> addCandidateToRoom({
    required String roomId,
    required RTCIceCandidate candidate,
  }) async {
    final roomRef = _db.collection(_roomsCollection).doc(roomId);
    final candidatesCollection = roomRef.collection(_candidatesCollection);
    await candidatesCollection.add(candidate.toMap()..[_candidateUidField] = userId);
  }
}

Set local description and start listening for an answer and other users candidates (step 4 - 5 from table):

import 'package:flutter_webrtc/flutter_webrtc.dart';

state.peerConnection.setLocalDescription(offer);

_firebaseDataSource.getRoomDataStream(roomId: roomId).listen((answer) async {
  if (answer != null) {
    state.peerConnection.setRemoteDescription(answer);
  } else {
		// stream return value is null means that call was ended and room was deleted
    emit(clearState);
  }
});
_firebaseDataSource.getCandidatesAddedToRoomStream(roomId: roomId, listenCaller: false).listen(
  (candidates) {
    for (final candidate in candidates) {
      state.peerConnection.addCandidate(candidate);
    }
  },
);

...

import 'package:cloud_firestore/cloud_firestore.dart';
import 'package:flutter_webrtc/flutter_webrtc.dart';

class FirebaseDataSource {
  final FirebaseFirestore _db = FirebaseFirestore.instance;

  static const String _roomsCollection = 'rooms';
  static const String _candidatesCollection = 'candidates';
  static const String _candidateUidField = 'uid';

  String? userId;

  Stream<RTCSessionDescription?> getRoomDataStream({required String roomId}) {
    final snapshots = _db.collection(_roomsCollection).doc(roomId).snapshots();
    final filteredStream = snapshots.map((snapshot) => snapshot.data());
    return filteredStream.map(
      (data) {
        if (data != null && data['answer'] != null) {
          return RTCSessionDescription(
            data['answer']['sdp'],
            data['answer']['type'],
          );
        } else {
          return null;
        }
      },
    );
  }

  Stream<List<RTCIceCandidate>> getCandidatesAddedToRoomStream({
    required String roomId,
    required bool listenCaller,
  }) {
    final snapshots = _db
        .collection(_roomsCollection)
        .doc(roomId)
        .collection(_candidatesCollection)
        .where(_candidateUidField, isNotEqualTo: userId)
        .snapshots();

    final convertedStream = snapshots.map(
      (snapshot) {
        final docChangesList = listenCaller
            ? snapshot.docChanges
            : snapshot.docChanges.where((change) => change.type == DocumentChangeType.added);
        return docChangesList.map((change) {
          final data = change.doc.data() as Map<String, dynamic>;
          return RTCIceCandidate(
            data['candidate'],
            data['sdpMid'],
            data['sdpMLineIndex'],
          );
        }).toList();
      },
    );

    return convertedStream;
  }
}

Joining Room

Check if room with ID exists in Firestore, if yes fetch offer object (step 6 from table):

import 'package:flutter_webrtc/flutter_webrtc.dart';

final sessionDescription = await _firebaseDataSource.getRoomOfferIfExists(roomId: roomId);

if (sessionDescription != null) {
	...
}

...

import 'package:cloud_firestore/cloud_firestore.dart';
import 'package:flutter_webrtc/flutter_webrtc.dart';

class FirebaseDataSource {
  final FirebaseFirestore _db = FirebaseFirestore.instance;

  static const String _roomsCollection = 'rooms';

  Future<RTCSessionDescription?> getRoomOfferIfExists({required String roomId}) async {
    final roomDoc = await _db.collection(_roomsCollection).doc(roomId).get();
    if (!roomDoc.exists) {
      return null;
    } else {
      final data = roomDoc.data() as Map<String, dynamic>;
      final offer = data['offer'];
      return RTCSessionDescription(offer['sdp'], offer['type']);
    }
  }
}

Initialize RTCPeerConnectionobject, set event handlers, add local stream tracks and set remote session description (steps 7 - 9 from table):

await _createPeerConnection();
_registerPeerConnectionListeners(roomId);

state.localStream.getTracks().forEach((track) {
  state.peerConnection.addTrack(track, state.localStream);
});

await state.peerConnection.setRemoteDescription(sessionDescription);

Create answer from peerConnection object, send it to Firestore and set is as local description for peerConnection (step 10 - 11 from table):

import 'package:flutter_webrtc/flutter_webrtc.dart';

final answer = await state.peerConnection.createAnswer();
await state.peerConnection.setLocalDescription(answer);
await _firebaseDataSource.setAnswer(roomId: roomId, answer: answer);

...

import 'package:cloud_firestore/cloud_firestore.dart';
import 'package:flutter_webrtc/flutter_webrtc.dart';

class FirebaseDataSource {
  final FirebaseFirestore _db = FirebaseFirestore.instance;
  static const String _roomsCollection = 'rooms';

	Future<void> setAnswer({
    required String roomId,
    required RTCSessionDescription answer,
  }) async {
    final roomRef = _db.collection(_roomsCollection).doc(roomId);
    final answerMap = <String, dynamic>{
      'answer': {'type': answer.type, 'sdp': answer.sdp}
    };
    await roomRef.update(answerMap);
  }
}

Start listening for the room creator candidates:

import 'package:flutter_webrtc/flutter_webrtc.dart';

_firebaseDataSource.getCandidatesAddedToRoomStream(roomId: roomId, listenCaller: true).listen(
  (candidates) {
    for (final candidate in candidates) {
      state.peerConnection.addCandidate(candidate);
    }
  },
),
_firebaseDataSource.getRoomDataStream(roomId: roomId).listen(
  (answer) async {
    // stream return value is null means that call was ended and room was deleted
		if (answer == null) {
      emit(state.copyWith(clearAll: true));
    }
  },
)

Displaying Streams with Render Objects

Now both users have local and remote streams, and somehow we need to set sources for our RTCVideoRenderer objects. It can be made in state listener like this:

import 'package:flutter_webrtc/flutter_webrtc.dart';

if (state.localStream != null || _localRenderer.srcObject != state.localStream) {
  _localRenderer.srcObject = state.localStream!;
}
if (state.remoteStream != null || _remoteRenderer.srcObject != state.remoteStream) {
  _remoteRenderer.srcObject = state.remoteStream!;
}

To display these streams in UI check out RTCVideoView widget.

That’s it! You are ready to create videochat in your app.