Introduction to Tensorflow.js

Tensorflow.js is a powerful library that makes Machine Learning accessible to everyone by allowing it to run in the browser. This eliminates the need for managing dependencies that are typically required in other ML projects.

Required Software

To build a model using Tensorflow.js, you need:

A text editor
A browser (Chrome is recommended)
An internet connection

Developing the Model

You'll be developing your own model using two documents: an HTML and a JS document. Essentially, you'll create a simple web page with JavaScript.

Creating the HTML Document

Start by creating an HTML document. This document must include links to Tensorflow.js and other relevant models. In this example, we'll use the MobileNet model and KNN-classifier. Here's the necessary code:

<html>
  <head>
    <script src="https://unpkg.com/@tensorflow/tfjs"></script>
    <script src="https://unpkg.com/@tensorflow-models/mobilenet@2.0.2"></script>
    <script src="https://unpkg.com/@tensorflow-models/knn-classifier"></script>
  </head>
  <body>
    <div id="console" style="height: 200px"></div>
    <video autoplay playsinline muted id="webcam" width="224" height="224"></video>
    <div>
      <button id="class-a">Add Object 1</button>
      <button id="class-b">Add Object 2</button>
      <button id="class-c">Add Object 3</button>
    </div>
    <script src="index.js"></script>
  </body>
</html>

JavaScript: The Core Logic Setup Webcam Function Create a setupWebcam async function to define a webcam object for training the classifier. It uses the Navigator.getUserMedia() to prompt the user for webcam access.

The App Function The app function is where the main logic resides. It loads the MobileNet model, ties the webcam stream to it, and uses the stream to make predictions.

Code for index.js Here's the full JavaScript code to be saved as index.js:

let net;
const webcamElement = document.getElementById('webcam');
const classifier = knnClassifier.create();

// Webcam setup
async function setupWebcam() {
  // ... [Webcam setup code]
}

// Main application logic
async function app() {
  // ... [App logic code]
}

app();

Training the Model Use the MobileNet model activation function to train the KNN model. Then, feed this activation into the KNN model for training and prediction. Display these values on the page, and notify the user if the confidence is below 50%.

Running the Application After creating and saving index.js in the same directory as index.html, open index.html in a web browser to see your first ML web app.

Note on Model Usage In this project, only one model is run at a time to predict the webcam's view. Training your KNN model will switch the prediction from the MobileNet Model to your trained KNN model.