The All Programmable SoC enters the maker arena

The maker sphere provides both a popular hobby for many and an inspiration for young people to follow careers in science, technology, engineering and math subjects. Many of the projects within this sphere contain an embedded processor, commonly a member of the Arduino or Raspberry PI families, to provide the intelligence required.


By Aaron Behmanand Adam Taylor, Xilinx                Download PDF version of this article


Both Arduino and Raspberry PI are supported by a development environment which provides a range of software libraries, modules and examples. These aid the developer to quickly and easily interface with a range of peripherals from cameras to accelerometers and motors. It is this ease of use which reinforces the popularity of these processors within the maker sphere.

Until recently makers have considered All Programmable SoCs to be outside the sphere and something for the more specialist engineer. This, however, is no longer the case with the introduction of Zynq-based boards like the ZynqBerry, Pynq and Snickerdoodle and software-based development techniques. These boards are fitted with devices from the Xilinx All Programmable Zynq-7000SoC family which combines dual ARM Cortex-A9 processors with programmable logic from the All Programmable Artix-7 FPGA range. This provides the ability to accelerate the functions within the programmable logic fabric to significantly increase system performance. Traditional development however has segmented the design of the programmable logic from the software development, requiring specialist development experience to implement programmable logic design. But this is no longer the case. When these boards are coupled with the latest development environments which allow the application to be defined entirely in software, they become very interesting for this sphere. Especially, as these development environments provide the ability to exploit the programmable logic without the need for the user to be a FPGA specialist, providing the best of both worlds.

Figure 1. Maker development boards: left to right Arduino, ZynqBerry, Pynq, Raspberry and Snickerdoodle.

 

There are two development environments which can be used to create applications for these Zynq-based development boards. The first of these is the SDSoC development environment which is eclipse based. This environment allows the development of the application in C or C++, and then to seamlessly move functions from running on the ARM Cortex-A9 processors for acceleration in the programmable logic. The SDSoC environment uses High Level Synthesis (HLS) to move the selected C function in to the programmable logic. Once the HLS has completed connectivity, framework is used to integrate the HLS module with the software application. Apart from the performance boost provided by the acceleration of the function now in the programmable logic, the process is transparent to the user. Moving functions between the processors and the programmable logic is extremely simple and controlled within SDSoC using the project overview. When it comes to operating systems, SDSoC supports Linux which is commonly used within the maker sphere along with a real-time operating system (FreeRTOS) and a bare metal approach.

The second approach is provided by Pynq, which provides a development framework based upon Python and Juypter notebooks. Both are executed on a Linux distribution running on the processors while the programmable logic has a defined overlay which provides connections to the peripherals provided on the Pynq. Within the Linux distribution to support the peripherals and hardware overlay, there is a defined Pynq package which allows us to directly interface with the peripherals using Python. As Pynq provides two PMOD interfaces this package delivers significant support for a range of PMOD such as ADC, DAC to ease the integration with the Python application. Within Pynq the programmable logic is loaded with one of many overlays to offer hardware acceleration, there are several open source overlays in addition to the provided basic overlay. The users can programme Pynq by connecting to the Juypter notebook server via a web browser. Once connected to this notebook, they can develop and document their Python applications to run on Pynq. The ability to use Python and directly interface to the PMODs using Python provides for a very powerful development platform.

Figure 2. Result of the image tracking application

 

Both development methodologies provide the ability to use open source embedded vision frameworks like OpenCV to perform embedded vision applications. These applications can use web cameras when the Linux distribution supports the USB Video Class, or specific cameras like the Raspberry PI camera supported by the ZynqBerry. OpenCV allows them to develop in either C/C++ or Python, using this framework they can quickly and easily implement complex image processing algorithms using the acceleration of Zynq programmable logic to provide significantly higher performance. These applications can process images and detect objects or faces and more, using this framework.

When it comes to implementing a simple object detection algorithm they can run Linux, Python and OpenCV on Zynq-based platforms. Let us examine what is involved with implementing a simple object tracking system using OpenCV and a web camera. The algorithm they will implement is:

1) Capture the first frame from the web cam. This first frame serves as the reference background frame. They will detect any changes that occur in the scene from this first frame.

2) Convert the colour space from RGB to grey scale. This is a commonly used image-segmentation technique used to create binary images. Image segmentation covers several techniques that divide an image into multiple segments often called super pixels. Segmentation allows for easier analysis of the segment contents. In their application, they can use thresholding to segment the background from the foreground. This will produce a binary image.

3) Perform a Gaussian blur on the image. The performance of many image-processing applications that detect objects or edges can be negatively affected by noise present within the grabbed frame. Blurring the image prior to further processing reduces noise within the frame and this technique is often used for image processing, particularly for edge-detection algorithms (for example, the Laplacian of the Gaussian edge-detection algorithm). The result of this operation becomes the reference image against which to detect a change.

4) Repeat steps 1 to 3 again, that is capture another image from the web cam, convert the colour space to grey scale and perform a Gaussian blur on the image.

5) Calculate the absolute difference between the reference frame and the most recent captured image.

6) Perform a thresholding operation on the absolute difference to create the binary image.

7) Perform a morphological operation to dilate the image to enlarge any differences.

8) Find the contours remaining in the binary image and ignore any contours with too small an area.

9) Draw a box around each of the contours detected and display the original captured image over the HDMI output.

Developers can run the resultant Python code either directly on ZynqBerry or within a Juypter note book on Pynq. What they see is an image like the one below which identifies the changes from the reference image with boxes highlighting the differences. This example demonstrates both the power and ease with which they can use Zynq to perform their embedded vision application, using familiar open source frameworks. The presented development environment demonstrates the ease with which developers can exploit the capability of several Zynq- based development boards. These development environments enable the user to create applications which execute on the processors within the processor but accelerate their designs using programmable logic.


Related


Slimming program for medical operating devices

Operating devices in the medical sector are not only subject to strict controls and requirements. Nowadays design demands are becoming more and more important for developers of medical HMI devices. De...

 

Perfect Motion Control For the Networked World

We live in a physical world where everything is connected. Trinamic transforms digital information into physical motion with accessible, flexible, and easy to use toolkits putting the world’s be...


New High-Performance Serial NAND: A Better High-Density Storage Option for Automotive Display

The automotive requirements: speed, reliability and compatibility. Winbond's high-performance serial NAND Flash technology offers both cost and performance advantages over the SPI NOR Flash typica...


President Tung-Yi talks about Winbond

Winbond is a leading specialty memory solution provider with a wide rage of product portfolio. Owned technology and innovation are our assets for our industry and our customers. Winbond we are high qu...


New Memory and Security Technologies for Designers of IoT Devices

Internet of Things (IoT) edge nodes are battery-powered, often portable, and are connected to an internet gateway or access point wirelessly. This means that the most important constraints on new I...


Winbond TrustMe Secure Flash - A Robust and Certifiable Secure Storage Solution

Winbond has introduced the TrustMe secure flash products to address the challenge of combining security with advanced process nodes and remove the barriers for adding secure non-volatile storage to pr...


Ultra-Low-Power DRAM: A “Green” Memory in IoT Devices

Winbond is offering a new way to extend the power savings available from Partial Array Self-Refresh (PASR), which was already specified in the JEDEC standard by implementing a new Deep Self-Refresh (D...


Polytronics Thermal Conductive Board (TCB) at Electronica 2018

This video introduce the basic product structure, advantage, and application of Polytronics thermal conductive board (TCB). Polytronics exhibit wide range of circuit protection products and thermal ma...


Arrow and Analog Devices strategic partnership and collaborative approach to provide solutions for our customers.

Mike Britchfield (VP for EMEA Sales) talks about why Analog Devices have a collaborative approach with Arrow Arrow’s design resources are key, from regional FAEs in the field to online des...


WE MAKE IT YOURS! Garz & Fricke to present the latest HMIs and SBCs at Electronica 2018

Sascha Ulrich, Head of Sales at Garz & Fricke, gives you a quick overview about the latest SBC, HMI and Panel-PC Highlights at electronica 2018. Learn more about the SANTOKA 15.6 Outdoor HMI, the ...


Macronix Innovations at electronica 2018

Macronix exhibited at electronica 2018 to showcase its latest innovations: 3D NAND, ArmorFlash secure memory, Ultra Low Vcc memory, and the NVM solutions with supreme quality mainly focusing on Automo...


ams CEO talks about their sensor solutions that define the mega trends of the future

In this video Alexander Everke, ams’ CEO, talks to Alix Paultre of EETimes about their optical, imaging and audio sensor solutions in fast-growing markets – from smartphones, mobile device...


Intel accelerated IoT Solutions by Arrow

Arrow is showing Intel’s Market Ready Solutions in a Retailer shop with complete eco environment. From sensors via gateways into the cloud, combined with data analytics, the full range of Intel ...


CSTAR - Manufacturers of cable assembly from Taiwan

CSTAR was founded in 2010 in Taipei, Taiwan. Through years of experience, we are experts in automotive products, LCD displays, LCD TVs, POS, computers, projectors, laptops, digital cameras, medical ca...


NXP Announces LPC5500 MCU Series

Check this video to discover the new NXP microcontroller LPC5500, the target application and focus area. Links to more information: LPC5500 Series: World’s First Arm® Cortex® -M...


Molex Meets Solutions at Electronica

These are exciting times in the electronics world as Molex migrates from a pure connectors company to an innovate solutions provider. Solutions often start at the component level, such as the connecto...


Alix Paultre investigates Bulgin's new optical fiber rugged connector range at Electronica 2018

Alix Paultre interviews Bulgin's Engineering Team Leader Christian Taylor to find out more about the company's new range of optical fiber connectors for harsh environments. As the smallest rug...


Cypress MCU and Connectivity are the best choice for real-world IoT solutions.

Cypress’ VP of Applications, Alan Hawse, explains why people should use Cypress for their IoT connectivity and MCU needs. Cypress wireless connectivity and MCU solutions work robustly and sea...


Chant Sincere unveils their latest High Speed/High Frequency connection solutions at Electronica 2018

Chant Sincere has been creating various of product families to provide comprehensive connection solutions to customers. USB Series Fakra Series QSFP Series Metric Connector Series Fibro ...


Addressing the energy challenge of IoT to unleash billions of devices

ON Semiconductor introduces various IoT use cases targeted towards smart homes/buildings, smart cities, industrial automation and medical applications on node-to-cloud platforms featuring ultra-low po...


ITECH, world leading manufacturer of power test instruments, shinned on electronica 2018

ITECH, as the leading power electronic instruments manufacturer, attended this show and brought abundant test solutions, such as automotive electronics, battery test, solar array simulator, and electr...


ITECH new series give users a fantastic user experience

ITECH latest series products have a first look at the electronics 2018, such as IT6000B regenerative power system, IT6000C bi-directional programmable DC power supply, IT6000D high power programmable ...


SOTB™ Process Technology - Energy Harvesting in Embedded Systems is Now a Reality

Exclusive SOTB technology from Renesas breaks the previous trade-off between achieving either low active current or low standby current consumption – previously you could only choose one. With S...


Power Integrations unveils their new motor control solution

In this video friend of the show Andy Smith of Power Integrations talks to Alix Paultre from Aspencore Media about their new BridgeSwitch ICs, which feature high- and low-side advanced FREDFETs (Fast ...


Panasonic talks about their automotive technology demonstrator

In this video Marco from Panasonic walks Alix Paultre of Aspencore Media through their automotive technology demonstrator at electronica 2018. The demonstrator highlights various vehicle subsystems an...