installing opencv

-
OK. We set up our raspberry pi. We built a car around it. We made a serviceable GUI for it. We toyed with PWM. Now what? Well, we spent some money on a good usb camera, so how about image processing? Again, something relatively new for me. There is a fantastic collection of code called opencv, so let's use that. "OpenCV (Open Source Computer Vision) is a library of programming functions mainly aimed at real-time computer vision." To install it on the Raspberry Pi is a little bit of work, but with some googling, and following directions, it worked. But given the pi's speed and limited RAM, anything intensive is probably best done on another box. Here are the install instructions that worked for me:
sudo apt-get install build-essential
sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev

mkdir opencv
cd opencv
wget -O opencv.zip https://github.com/opencv/opencv/archive/3.4.1.zip
unzip opencv.zip

wget -O opencv_contrib.zip https://github.com/Itseez/opencv_contrib/archive/3.4.1.zip
unzip opencv_contrib.zip


cd opencv-3.4.1/
mkdir build
cd build

cmake -DCMAKE_BUILD_TYPE=RELEASE \
    -DCMAKE_INSTALL_PREFIX=/usr/local \
    -DINSTALL_C_EXAMPLES=ON \
    -DINSTALL_PYTHON_EXAMPLES=ON \
    -DOPENCV_EXTRA_MODULES_PATH=~/opencv/opencv_contrib-3.4.1/modules \
    -DBUILD_EXAMPLES=ON \
    -DBUILD_DOCS=ON  ..

make
sudo make install
sudo ldconfig

# now install matplotlib:
sudo apt-get install python3-matplotlib
Be warned that 'make' took 7 hours on my raspberry pi. So it was important to power the pi from the micro-USB cable rather than the battery pack on the car. For a speed up, it is possible to run jobs in parallel, eg: 
make -j4
but on the pi you run the risk of running out of RAM. So I opted for the slower version.

Now on to some elementary image processing. First up, average images from our camera. The output from the camera can be quite good, but is noisy in lower light levels. Average tidies this up a bit: 
import numpy as np
import cv2

count = 20

cap = cv2.VideoCapture(0)

# create a list of first count frames:
img = [cap.read()[1] for i in range(count)]

# rotate images:
rot_img = [cv2.flip(i, -1) for i in img]

# convert all to float64:
float_img = [np.float64(i) for i in rot_img]

# find the average:
mean_img = float_img[0]
for k in range(1, count):
    mean_img += float_img[k]
mean_img /= count

# Convert back to uint8:
int_mean_img = np.uint8(np.clip(mean_img, 0, 255))

# show output:
cv2.imshow('initial', rot_img[0])
cv2.imshow('average', int_mean_img)

# save output:
cv2.imwrite('raw-image.png', rot_img[0])
cv2.imwrite('ave-image.png', int_mean_img)

# tidy up and quit:
cv2.waitKey(0)
cv2.destroyAllWindows()
Next up, unscaled and scaled image similarity. These two functions return 0 for completely distinct, 1 for identical, and values in between otherwise. First, unscaled: 
import numpy as np
import cv2

def image_simm_unscaled(im1, im2):
    # convert all to float64:
    fim1 = np.float64(im1)
    fim2 = np.float64(im2)

    wf = np.sum(np.absolute(fim1))
    wg = np.sum(np.absolute(fim2))
    wfg = np.sum(np.absolute(fim1 - fim2))

    if wf == 0 and wg == 0:
        return 0
    return (wf + wg - wfg) / (2 * max(wf, wg))
Now scaled: 
import numpy as np
import cv2

def image_simm_scaled(im1, im2):
    # convert all to float64:
    fim1 = np.float64(im1)
    fim2 = np.float64(im2)

    s1 = np.sum(fim1)
    s2 = np.sum(fim2)

    if s1 == 0 or s2 == 0:
        return 0

    wfg = np.sum(np.absolute(fim1 / s1 - fim2 / s2))

    return (2 - wfg) / 2
Next, combine average and image-similarity into one function, simm-average. The idea being, we only add it to our average-image if it passes a similarity threshold. This helps with the ghosting problem I had with a simple average, but seems not to 100% solve that issue. Maybe with tweaks to the value of IMAGE_SIMILARITY we could completely remove ghosting?
IMAGE_SIMILARITY = 0.75

def create_simm_average_camera_image(count):
    # create a list of first count frames:
    img = [cap.read()[1] for i in range(count)]

    # rotate images:
    rot_img = [cv2.flip(i, -1) for i in img]

    # convert all to float64:
    float_img = [np.float64(i) for i in rot_img]

    # find average of similar images:
    mean_img = float_img[-1]
    i = 1
    for k in range(1, count):
        tmp_img = float_img[k]
        r = image_simm_scaled(mean_img, tmp_img)
        if r > IMAGE_SIMILARITY:
            mean_img += tmp_img
            i += 1
    mean_img /= i
    print('we averaged %s images' % i)

    # Convert back to uint8:
    int_mean_img = np.uint8(np.clip(mean_img, 0, 255))

    return int_mean_img
In the next post, something more interesting, edge-enhance.

Comments

Post a Comment

Popular posts from this blog

introducing active-map

-
Image
Let's keep tweaking our GUI. I decided to implement what I call an active map. Code below. Instead of changing the camera angle by clicking up 10 degrees, left 10 degrees, up another 10 degrees, how about we have a map. The map covers the full 0-180 * 0-180 angle range (of the two camera servos), and you click on the location you want the camera to move to. After an afternoon of coding, I now had: I'll dive straight into the code. First we need these two helper functions: def constrain(val, min_val, max_val): val = max(val, min_val) val = min(val, max_val) return val def num_map(value, fromLow, fromHigh, toLow, toHigh): return (toHigh-toLow)*(value-fromLow) / (fromHigh-fromLow) + toLow constrain ensures val is in the range min_val <= val <= max_val, while num_map maps val from the range [fromLow, fromHigh] to the range [toLow, toHigh]. And here is our new active-map class: class ActiveMap(): def __init__(self, name, width, val, maxi, mini, xpos
Read more

using edge-enhance

-
Image
Last post we installed opencv on our car. This post we put it to use with edge-enhance. First, we need some decent test images. Well, thanks to my panorama-scan, which I will discuss in my next post, I have some. Let's jump right into the images, and then give the code later. So, we have this image: Now we edge enhance it: Then we edge enhance it in grayscale mode: Then we have this image: Now we edge enhance it: Then in grayscale mode: It works OK I suppose. I wonder if we can improve it? Anyway, here is my code , inspired by this code : import sys import numpy as np import cv2 iterations = 10 def massage_pixel(x): if x < 0: x = 0 # x *= 20 x *= 30 # x *= 3.5 x = int(x) if x > 255: x = 255 return 255 - x def main(argv): filename = "ave-img.png" img_codec = cv2.IMREAD_COLOR grayscale = False if argv: filename = sys.argv[1] if len(argv) >= 2 and sys.argv[2] == &quo
Read more

panorama-scan

-
Image
In this post I introduce the panorama-scan code. Once you put a decent quality camera on the car, the camera servo is perfect for creating panoramic scans. There was a slight problem with this idea. You need to be logged in to the car to start and stop code. How can we set the car to scan when a long way from your home wifi network? I opted for a script that starts up at boot-up, and auto-runs the panorama-scan if it is not associated with a network, and does nothing if it is. The idea being if it is in wifi range of my network, I don't need the panorama-scan to start by itself. So with this setup, I take my car to some scenic spot, power it up, wait for it to scan. Power down, move to the next spot. And so on. Here is our start-up script : #!/bin/bash # wait a while, hopefully long enough for network to start: sleep 2m # check if we can reach google DNS: nc -zw 1 8.8.8.8 53 >/dev/null 2>&1 online=$? # NB: raspberry pi time is only updated when connected to a netw
Read more