Back-end Tech Stack for Custom eLearning Solutions

Some time ago we received a request from a client who wanted to create an online learning platform. The platform had to include both real-time webinars and on-demand video streaming functionality. The client contacted us after receiving an estimate from other developers which exceeded his budget. Thus, our main goal was to find a solution that offered existing modules for this project. 

We conducted research and found that the Google Cloud Platform we are partnering with, offered special components for projects with video streaming, as well as a bunch of other handy functions, including analytics and cloud data storage. 

We’ve already talked about how to create an online learning platform and essential features for the eLearning platform. In this article, we share our technological expertise in implementing GCP’s components for a custom online learning platform development. 

But before diving into tech details, let’s find out what an e-learning platform consists of. 

Online learning marketplace architecture and main components

The online education marketplace should consist of four main solution components, closely connected to each other. They are:

• A web-based application that will contain most of the features to simplify the development process of the education marketplace
• Video processing for transcoding, uploading, and streaming educational videos on-demand
• Data processing that will receive events from other parts of the custom eLearning service and ingest them
• Cloud storage for storing original uploaded on-demand education video files

[eLearning marketplace architecture and main components by The APP Solutions]

With this in mind, let’s move further and find out more about each architecture component and the technologies to implement them. 

Educational software from scratch: essential tech stack

A web application is a part of an online learning platform that includes most features, such as user authentication, user management, and others. Most of the functionality will be integrated into web applications using API. 

Architecture

When developing an e-learning web app, you can choose among the following options: 

• Building a monolithic eLearning application if you’re considering developing a learning platform software with a limited amount of content, users, etc. In other words, you are not going to significantly scale it in the future. 
• Splitting eLearning apps back-end into microservices is an excellent choice if you are going to create an online learning platform where third-party educational organizations will post and sell their online courses. 

[Source: Medium]

While monolithic app architecture significantly simplifies initial educational marketplace development, it could become too large with time, and hence, difficult to manage, while microservices architecture allows updating, deploying, and scaling each element independently. As for the web frameworks, we suggest using Symfony4 (PHP) or Django (Python). 

Video service

Video service allows both you and other users to upload video materials to the platform and sends transcoded videos to user devices. As you may know, the quality of streaming video depends on many factors, such as video formats, mobile devices operation memory, slow and fast internet speeds. To provide users with high-quality video streaming, video service transcodes video files and creates multiple versions of the same video in different sizes, thus making it possible for users to watch online courses even with slow Internet while consuming a small amount of the user’s device memory. 

WebRTC server

WebRTC stands for Web Real-Time Communication. This component of an e-learning platform enables platform users to arrange video streaming sessions. WebRTC is an open-source framework with a library that is available via a web browser. This tool also supports native platforms such as iOS and Android. The developers will add and set this component via an application programming interface. As an alternative, you can also use Kurento or Janus open source products. 

Video Processing tech stack 

This component of eLearning platform architecture is responsible for transcoding both pre-recorded and live streaming video lessons. Now, let’s take a closer look at its tech stack. 

Transcoding service 

This service transcodes videos uploaded to the platform by you or other users. The service uses the FFmpeg library to transcode uploaded videos, which is open-sourced. 

The transcoding service’s primary function is to decompress the raw files and convert them into different compressed formats. In this way, your learning platform will provide fast, efficient streaming over the Internet right into a desktop or mobile client. 

The main reason we suggest the FFmpeg library is that traditional transcoding services require many processors, a lot of memory, and a lot of storage. For example, it takes more than an hour to transcode an hour’s worth of high-definition video. At the same time, the FFmpeg library allows transcoding such videos in real-time. 

[Source: Intel] 

Live transcoding service 

Live Transcoding Service receives a Real-time Transport Protocol stream from WebRTC (Web Real-Time Communication) server. It transcodes it to HTTP Live Streaming (also known as HLS), an HTTP-based adaptive bitrate streaming communications protocol. In other words, it allows teachers to stream live lessons. 

But why does your platform need it? Let’s explain. 

Imagine you want to do a live broadcast using a camera and encoder. You’ve already compressed your video with an RTMP encoder and selected the H.264 video codec at 1080p. But if you attempt to stream it directly, users will not see it. Thus, a live transcoding service is essential to do a live broadcast to all users with slower data speeds, tablets, mobile phones, and connected TV devices, allowing playback on almost any screen on the planet. Thus, without using this service, viewers without sufficient bandwidth can’t view the stream. 

[Source: Wowza]

Data Processing tech stack 

Streaming ETL

Streaming ETL processes the movement of real-time data from one place to another. This element will be connected with both analytics and real-time databases. 

ETL means database functions, such as extract, transform, and load. Let’s look at these functions in more detail. 

• Extract means collecting data from some source, which will be in other parts of the system. 
• Transform means any processes performed on that data
• Loads refer to sending the processed data to a destination, such as analytics and real-time databases. 

For the streaming data processing pipeline, we suggest using Cloud Dataflow to receive events from other parts of the system and ingest them into BigQuery, responsible for data analytics. 

Storage tech stack 

The storage of your learning platform will include four main elements. They are the Main database, analytics, file storage, and Real-time database. Let’s check the technologies required for each component. 

Main database

The main database will organize data items into columns and rows. This component ensures all co-dependent elements of your platform are working simultaneously.  

Since all other platform components are located on Google Cloud Platform, we suggest using relational databases, such as the CloudSQL database as the central database for a web application. CloudSQL makes it easy to set up, maintain, manage, and administer your relational databases on the Google Cloud Platform. This database supports MySQL, PostgreSQL, or SQL Server. 

Analytics 

This element gathers all information about the actions happening on your platform and analyzes them to give you handy insights about your platform users. Since your platform will include a massive amount of data, Google Analytics does not suit this task. Therefore, we suggest applying BigQuery Analytics to handle vast amounts of data to analyze it in a more streamlined way. BigQuery Analytics also allows storing necessary data of special events in separate spreadsheets. For example, thanks to BigQuery Analytics, you can receive a detailed report on the most popular courses, how many users bought paid subscriptions, how many users access your platform via a web browser,  and the number of mobile device users. This tool will also be handy when you start running advertising campaigns and need to find out the most effective advertising channel.

File storage

In the file storage, you will store all the materials used by your learning platform. These materials will include uploaded video files, transcoded video files, images, etc. Since on-premise solutions do not suit a streaming platform’s growing needs, we suggest applying Google Cloud Storage for all your files. Moreover, Google Cloud Storage enables low-latency content delivery across the globe, thanks to geo-redundant storage with the highest availability and performance level.

Real-Time database

For storing videos that users are watching, your eLearning system needs a real-time database. For this purpose, we suggest leveraging The Firebase Realtime Database. It is a cloud-hosted NoSQL database that allows storing and synchronizing data between users in real-time. Thanks to Android, iOS, and Javascript SDKs of Firebase, your users can access your online courses via web browsers, iOS, and Android devices. Besides this, this database supports offline data access. 

Now that we’ve examined the essential components of an eLearning platform’s back-end, let’s find out how much this project will cost. 

Related reading: 

Case Study: Video Streaming App Proof of Concept

How much does it cost to create your own eLearning platform?

Despite most of your platform’s back-end components being open-source, you still need to pay a development team that will implement them. Besides this, developing a custom online learning marketplace still requires front-end, design, testing, and project management services, for which you must also pay. 

So, let’s check out how much money you need to receive a fully-fledged eLearning platform. 

• Back-end requires from 800 to 1000 hours 
• Front-end from 300 to 500 hours 
• Design from 250 to 400 hours 
• QA around 1000 hours 
• Project management from 2500 to 3500 hours 
• Project documentation will take from 650 hours 

To receive a more detailed estimate of your eLearning marketplace, use our project cost calculator. 

eLearning tools and technologies: Final word

The right back-end architecture components are essential for sophisticated projects, such as in the eLearning marketplace. The choice of back-end components will impact the quality and downtime of both videos on-demand and live streams. 

Thanks to cloud providers like Google Cloud Platform, developers can leverage the best software for creating an eLearning project’s back-end, which significantly reduces the time and money required to deliver the project. 

What our clients say