Building ClipJS: a video editor that works completely in the browser
Introduction
This is the introduction for why I wanted to start such a thing in the first place (you can skip it if you want).
So, I’ve been using Ubuntu as my main OS for about a year now, and at some point, I wanted a simple video editor. Nothing fancy like Premiere just something lightweight to cut and stitch clips. I tried OpenShot Video Editor, but for some reason, every time I exported, I ended up with a black screen (with audio in the background). I also tried a few other editors that were less popular, but I ran into different annoying issues. I was too lazy to really dig in and fix them, I mean, the vids were really simple, trimming and adding some texts, nothing fancy, I didn’t bother digging deeper and just lived with it.
Then, during a discussion with a friend, he mentioned there is no good online video editor out there. Well, I didn’t even know such a thing existed. Turns out, there are quite a few, and it’s actually a really cool concept like photo converter tools or those websites that quickly turn images into PDFs. You don’t have to install a full app just for a tiny task; you just want something quick and ready to use.
But here’s the problem: most of them run everything on the server, so you have to upload your clips first. (Not a great thing when you are in a country with a limited internet quota) .
While searching, I found out there are some browser-based ones that were even created by indie developers, which really inspired me, but actually, most of them were closed source. Around this time, I was also waiting for Google Summer of Code results, and I wanted to keep myself busy building something special, not just another CRUD app. So I started and decided to make it open source.
So, diving into the technicalities, the project can be split into three parts: preview rendering, UI & state management, and final rendering.
Preview Render
At first, I tried to build the preview player using the Canvas API. But I quickly realized it would take too much time to be optimized and work properly without bugs. While searching, I came across a library called Remotion that does what I want in really simple steps, so why reinvent the wheel?
- Composition
Here, define your video scene as a React component. This is where you place text, images, videos, animations, etc. - Mount it in a Player
Remotion provides a <Player> component that mounts your Composition and makes it behave like a normal video. - Frame Rendering
The Player calculates the current frame based on time and FPS, renders your Composition for that frame, and shows it in real time. - Playback Controls
You get APIs like play, pause, seek, and loop to control playback and customize how your video behaves.
UI & state management
For state management, I created two IndexedDB databases:
- One for uploaded files.
- One for project details (where media files are referenced by ID so the actual file is only stored once even if it’s reused).
I used Next.js and Tailwind as this is the stack I’m most comfortable with. I didn't want to have anything fancy just something simple and straight to the point, which is the style I like.
Exporting the video
Remotion gives you an export option, but it requires a Node.js server. I wanted to keep everything browser-side, so I stuck to my original plan with FFmpeg. At first, I released the editor without rendering support, just to see what people thought. The feedback was super positive, which motivated me to keep pushing.
Eventually, I went back to FFmpeg, I also learned a lot about FFmpeg, encoding, and decoding media from here. I handled trimming, scaling, text overlay, etc, with filter_complex. This approach required re-encoding all video and audio tracks, not the most optimal solution, but it’s the one I’ve been using up until writing this.
Conclusion
At the end, it has been an amazing journey. I learned a lot, gained some recognition, and even now, 4 months after launch, I still get around 500 visitors a month. which means some people actually find it useful. I even got pull requests from others, which is really cool. I’m still working on ClipJS in my free time, and it continues to be a project I enjoy building.