Simplicity of Python, speed of C and Extensibility of Lisp. Nim is a new contender in the programming language industry which has made bold claims. Learn about Nim by building a Drift Detector in OpenCV

Table of contents

1. Introduction
2. Some Syntax
3. Drift Detection
4. Deeper Look

Introduction

Traditionally, integrating C++ libraries with any other language has been a challenge mainly due to the fact that C++ code is mangled, unlike C where names like function names or variable names are directly converted to symbol names in the compiled output, in C++ every name is mangled which allows for function overloading and namespacing.

Due to this fact (which allows for function overloading) we don't have a deterministic naming convention for calling some compiled C++ functions in some other languages. Instead of having manual bridges and cumbersome bindings, Nim's unique approach to interoperability offers a refreshing solution. It is almost like we are templating C++ in Nim, An Absolute Delight !!

Some Syntax

Nim borrows a lot from python's syntax like using indentation to define blocks and using whitespaces instead of brackets to define arguments.

# fibnonacci sequence generator in nim

proc fibonacci(n: int): seq[int] =
  var fibSeq = @[0, 1]
  for i in 2..n:
    fibSeq.add(fibSeq[i - 1] + fibSeq[i - 2])
  fibSeq

let n = 10
let fibSeq = fibonacci(n)
echo "Fibonacci sequence up to ", n, ":", fibSeq

Read more about the similarities and how a python developer can transition to Nim here

That's where all the similarities end, Nim is compiled rather than being interpreted and is statically typed, but we won't be going too deep into the general syntax rather how we can use C++ in Nim

Pragma

Pragmas are Nim's method to give the compiler additional information, we use Pragmas extensively when interoping with C/C++ . Let's look at some important Pragmas

Link

{.link: "<path/to/library>".}

This Pragma specify the paths to the library that needs to be linked during compilation

Import Type Definition

type MyInteger {.importcpp: "long".} = int

In Nim, a type represents a set of values along with operations that can be performed on those values.

In this declaration:

• MyInteger is the name of the Nim type we are defining.
• importcpp: "long" specifies that MyInteger corresponds to the long data type in C++.
• = int indicates that MyInteger in Nim is represented as an int, which is Nim's arbitrary-sized integer type.
• The . at the end of the pragma {.importcpp: "long".} signifies the end of the pragma declaration.

here we have established a mapping between the long type of C++ and the int type in Nim

A more complex example

type myMat* {.importcpp: "cv::Mat", header: "<opencv2/core.hpp>",
   byref.} = object
  cols: int
  rows: int

here we are interfacing the opencv libraries and importing the cv::Mat data type

Here's a breakdown of the declaration:

• myMat is the name of the Nim type we are defining.
• importcpp: "cv::Mat" specifies that myMat corresponds to the cv::Mat class in C++, which is a fundamental structure in OpenCV for representing matrices upon which we do operations like rotation .
• we used the header pragma here as header: "<opencv2/core.hpp>" which indicates the header file in which the cv::Mat class is declared. .
• byref denotes that instances of myMat are passed by reference, meaning that they are accessed through a reference rather than being copied. This is important for efficiency when working with large data structures
  • it will become clear later what would happen if we don't pass the byref

Additionally, within the myMat object:

• cols and rows are integer fields representing the number of columns and rows in the matrix, respectively. These fields provide access to the dimensions of the matrix within Nim code.

internally doing myMat.cols in nim is translated to cv::Mat::some_matrix.cols in C++

⚠️ we have only imported the data type and not the methods that are defined on this type including the constructor itself

Side Note: The * in nim doesn't represent a pointer object rather it is how we define a public symbol which can be accessed from different nim files, similiar to how we do pub in rust

The code won't work as it is because we are not linking against the opencv libraries. Remember to link we use the link pragma l ike

{.link: "/opt/Headers/opencv-4.9.0/build/lib/libopencv_core.so.4.9.0".}

Importing Constructors

proc myMatConstructor*(cols: int, rows: int, types: int): myMat {.importcpp: "cv::Mat::Mat(@)",
                                                  header: "<opencv2/core.hpp>", constructor.}

• proc myMatConstructor* declares a procedure named myMatConstructor.

• (cols: int, rows: int, types: int) specifies the parameters that the constructor expects. These parameters represent the number of columns, rows, and the type of the matrix, respectively.

• : myMat indicates that the constructor returns an instance of the myMat type which we defined earlier

• {.importcpp: "cv::Mat::Mat(@)"} specifies the C++ constructor that the Nim procedure corresponds to. In this case, it corresponds to the constructor of the cv::Mat class in OpenCV, which initializes a matrix with the given dimensions and type.

  • the @ symbol means that put whatever arguments were passed to procedure here

• header: "<opencv2/core.hpp>" specifies the header file in which the C++ constructor is declared. This ensures that the necessary declarations are available to the Nim code.

• constructor denotes that this procedure is a constructor, ensuring that it's called automatically when a new instance of the myMat type is created.

Importing Procedures

proc at*(mat: myMat, x: int, y: int): int
  {.importcpp: "at<uchar>", header: "<opencv2/core.hpp>".}

The at procedure allows us to access elements of a matrix represented by the myMat type which internally is mapped to the cv::Mat type

• proc at*(mat: myMat, x: int, y: int): int: This defines a procedure named at that takes three parameters: mat, which is an instance of the myMat type representing the matrix, and x and y, which are the coordinates of the element we want to access. The procedure returns an integer value.

• {.importcpp: "at}: This pragma specifies the C++ function or method that the Nim procedure corresponds to. In this case, it corresponds to the at function/method in the C++ codebase, which is used to access elements of the matrix.

  • see how we have passed the C++ char type inside the <> we can change it to something else like at<int>

We see that we can write pure C++ inside the "" after the .importcpp pragma and wrap it in a nim procedure. You can have multiline C++ code inside those quotes and it will still work, that's what I love about nim It suprisingly just works

Note: inside the header field you have to pass the full path to the library, if you don't want to do that then you can tell the Nim compiler where to look for the headers. There are multiple ways but one way is by using a $project.nim.cfg file like here and don't forget to enable the C++ backend in Nim by adding the command line argument or in the nimble config

Drift Detection

Let's put all this syntax used into making a Drift AKA Motion detector. Motion detection is a crucial component of computer vision systems that involves identifying and analyzing changes in the position or appearance of objects within a scene over time.

there are multiple ways of detecting motion

• Frame Differencing
• Optical Flow
• Background Subtraction

we will be focusing on Frame Differencing which is the easiest one to implement.

Consider a black and white image as an array of pixels, each with a value ranging from 0 to 255. When capturing two images, one at time 0 seconds and the other at 1 second, any motion present between the two instances results in a change in the pixel configuration. This change manifests as non-zero differences between corresponding pixels in the first and second images.

Consider two grayscale images represented as 2D arrays:

Image at time t (0 seconds):

[[100, 120, 110],
 [105, 125, 115],
 [95, 115, 105]]

Image at time t+1 (1 second):

[[100, 125, 110],
 [105, 120, 115],
 [95, 115, 105]]

Differences:

[[0, 5, 0],
 [0, -5, 0],
 [0, 0, 0]]

The matrix is not all zeroes so we have detected some motion We will build upon this Frame Differencing method our Motion Detector

Coding

For the input frame sequence let's use the webcam of our computer. You can access it in opencv C++ using cv::VideoCapture::open(0) which will use the default camera interface

First we define a type which holds the cv::VideoCapture type

type myVideoCapture* {.importcpp: "cv::VideoCapture",
    header: "<opencv2/videoio.hpp>".} = object

and now we define the open function for it

proc open(cap: myVideoCapture, index: int): bool {.importcpp: "cv::VideoCapture::open",
    header: "<opencv2/videoio.hpp>".}

see how Nim handles the return value mapping from a C++ function to the Nim procedure , we didn't have to define what the function should return when some value X is returned from the C++ function, its piped to Nim bool data type.

Now we use the cv::VideoCapture::read method to read from our webcam.

proc read(cap: myVideoCapture, frame: myMat): bool {.importcpp: "cv::VideoCapture::read",
    header: "<opencv2/videoio.hpp>".}

Let's test this code by doing

var cap: myVideoCapture
open(cap, 0)
var colored_frame: myMat

if read(cap, colored_frame):
	imwrite(colored_frame,"output.png")
else:
    {.fatal: "not able to read from capture device".}
    discard

the fatal pragma can be used to output errors, Nim provides a lot of pragmas that allow for customizability and flexibility

Anyways, if everything was implemented correctly the program will read a frame and then save it to a png file.

Now that we can read and save frames let's move onto doing the frame differencing.

The above operations have been using data structures such as cv::Mat for image representation. While it's possible to perform the motion detection operations directly on cv::Mat objects, working with Nim's native data structures like 2D arrays can often lead to cleaner and more readable code.

to convert a cv::Mat into a Nim 2D array we do

proc at*(mat: myMat, x: int, y: int): int
  {.importcpp: "at<uchar>", header: "<opencv2/core.hpp>".}

proc matToMatrix(mat: myMat): seq[seq[int]] =
  var matrix: seq[seq[int]] = @[]
  for i in 0..mat.rows-1:
    var row: seq[int] = @[]
    for j in 0..mat.cols-1:
      row.add(at(mat, i, j))
    matrix.add(row)
  return matrix

• The at procedure allows us to access elements of a matrix represented by the myMat type which internally is mapped to the cv::Mat type
• seq is Nim's vector representation
  • we initialises it using the @[] which initialises an empty 2D vector
• We iterate the cv::Mat type and fill up the 2D sequence of ints

Ok so we got the 2D representation of the frame now what ? We take 2 frames taken one after the other and create a new difference frame out of it.

proc diffMatrix(f1: seq[seq[int]], f2: seq[seq[int]]): seq[seq[int]] =
  var diff_matrix: seq[seq[int]] = @[]
  for i in 0..f1.len-1:
    var row: seq[int] = @[]
    for j in 0..f1[0].len-1:
      if(abs(f2[i][j] - f1[i][j]))>0:
        row.add(abs(f2[i][j] - f1[i][j]))
      else:
        row.add(0)
    diff_matrix.add(row)
  return diff_matrix

simple enough eh ? we it erate both the matrices and add the difference to the diff_matrix, if there is none we just add a 0

now we just iterate the diff_matrix and check if its all zeros or not

proc isMotion(img: seq[seq[int]]): bool =
  for i in 0..img.len-1:
    for j in 0..img[0].len-1:
      if img[i][j] != 0:
        return false
  return true

and add some helper functions to output if we have detected motion or not

while true:
  var frame1: myMat
  discard read(cap, frame1)
  os.sleep(100)
  var frame2: myMat
  discard read(cap, frame2)
  var te1: seq[seq[int]] = matToMatrix(g1)
  var te2: seq[seq[int]] = matToMatrix(g2)
  var drift = diffMatrix(te2, te1)
  if isMotion(drift):
	  echo "Motion Detected"

Voila !!! we have made a motion detector in Nim and interoping with C++

A Deeper Look

How is this happening ? if C++ does name mangling then how does Nim know everytime what the symbol name is going to be ? What sorcery is this 👾

Under the hood Nim is actually compiling the code we wrote to C++ and then interfaces with the other C++ normally. Pretty Cool isn't it !!

You can take a look at the C++ files generated by Nim inside /.cache/nim/$projectname(_r|_d) directory

Final Regards

Nim really is a great language, the python like readability the speed of C/C++ and impressive interoperability makes it a really compelling language to learn and build projects in.

In Rust/ Zig or any other language you would have modify the C++ code and add a bunch of extern C calls so that the name of that symbol isn't mangled but in Nim it just works.

Helpful links

• Source code available here