Raspberry Pi as a Camera Development Kit for computer vision and machine learning (Introduction)

Market opportunity

I have been looking for some time for a camera to complement my smart home and I came to the conclusion that there is no product in the market that provides a decent solution for the user. This are some features offered by cameras currently in the market:

  • Store video in the cloud or locally but very hard to use
  • Detect motion
  • Night Vision
  • Send notifications

All this features are standard now and while companies have been focusing in charging annual fees for storing your video in the cloud, as a premium service, there is nothing noble about this and since this is no diferenciador they are doom to fail as soon as a more appealing product jumps into the market since there is no loyalty besides the money you spent in a device that can’t do anything if the cloud service they depend off is deprecated.

http://www.businessinsider.com/googles-nest-closing-smart-home-company-revolv-bricking-devices-2016-4?r=UK&IR=T

If you think of market opportunity clearly the industries that will have a huge wrought in the following years are IoT, autonomous cars, virtual reality and robotics.

Interesting enough, the thing they all have in common is that they all need cameras.

Even if one flops, for example VR, there will still be 3 great opportunities. From an investment perspective seems like a great portfolio.

http://fortune.com/2016/02/24/robotics-market-multi-billion-boom/

http://www.marketsandmarkets.com/PressReleases/home-automation-control-systems.asp

http://www.investopedia.com/articles/investing/120215/virtual-realityaugmented-reality-markets-set-explode-future.asp

http://spectrum.ieee.org/cars-that-think/transportation/self-driving/self-driving-cars-market-2030

 

Product Overview

Camera OS is more than a camera firmware, it is a whole platform. It will not only take care of connectivity but allow the device to run applications that use the camera in many exciting new ways making it the smartest camera in the market that never stops to learn.

Cameras running this OS can empower developers and users with the ability to run different application that can adapt to specific needs and privacy concerns giving the to opportunity to customize conditions that can trigger notifications to devices or services.

Combined with à la carte cloud base premium services and new technologies like machine learning cameras running this OS have the potential to be the go to camera to solve every days needs in terms of security, research, convenience and entertainment.

What makes this combination of software and hardware so innovative is the ability of processing and analyzing video locally instead of having to stream it to the cloud before any analysis is done to it.

In the following pages I will describe some of the potential use cases for Camera OS based devices and this is only the beginning. Completely open to developers the opportunities are only limited by the imagination and can spark a completely new way of conducting studies and research, managing security and even entertainment.

Below are some examples of uses that will be available on day 1.

Smart Camera as a security device

Cameras are essential if you think about security. While almost every camera can record video and detect motion there isn’t much more that the camera can do and even this is limited.

One example is motion, motion is the most popular feature on security cameras but how about if you have a pet? What if I tell you that this camera can distinguish between a people and your beloved companion. It can even identify a specific person if we train the device to recognize faces. 

caballo

Smart Camera as a convenience device

Imagine the camera as an infinite resource of information to your Smart Home. This device would be the ultimate sensor for the house. The possibilities are endless.

licence

Open garage door based on licence plate recognition.

darkroomn

Turn lights on when a person is in the room but not a shadow like other cameras that detect pixel changes as motion.

Keeping the lights on if there is no motion but a person is in the room is a constant struggle for motion sensors. The perfect example is when you watch TV, usually you don’t move much and the motion sensor perceives this as no motion sending a command to your house to turn the light off.

tv

Notify  me when my wife gets home or start my favorite playlist when I arrive.

face

This feature was part of Mark Zuckerberg new year resolution. https://www.facebook.com/zuck/posts/10102577175875681

people

Count people in a room.

We can keep going on and on about uses for cameras in our daily life. Imagine how many new uses developers will create and share with other users.

Smart Camera as an entertainment device

Millennials are all about sharing. This camera allows you to share your party directly in your Facebook feed.

facebook

http://www.usatoday.com/story/tech/news/2016/04/06/facebook-adds-features-live-video-push/82680084/

youtube

How about streaming your DIY projects live on youtube?

Smart Camera and Privacy

Other opportunities for camera is to use them as sensors. Cameras in the house can be a big privacy concern. Now, imagine a camera that is aware of the environment but doesn’t stream video at all. Because this camera can process video locally you can turn off the streaming capability and still send updates of what is going on. For example, you can detect a person in the room and send a notification or an API, zigbee or bluetooth without having to share or stream any video or image.   samsung

I made a functional prototype that I showed at Samsung Developer Conference 2016 where I demonstrated how to integrate this camera to smartThings and how it can interact with the connected home.

Developer Kits

The developer kit camera is essential to reach developers and propel the ecosystem.

This is a cheap camera able to run applications ideal for developers and even home use. This camera would be capable of running all the scenarios described above.  

The use your own camera developer kit enables unique and more complex development. With the same power for computer vision it allows developers to use their own camera opening doors to scientist and researchers to connect all this new potential to their own field.

Some examples could be microscopes for finding patrons in viruses or bacteria, telescopes scanning the cosmos 24/7  and Infrared cameras for security, rescue missions and monitoring buildings and infrastructure.

Capture

Advance Developer Kit has a 4k camera and a bigger CPU and GPU that enables the most extreme and low latency use cases where time and amount of information are crucial. Some examples could be people tracking for both self driving cars and Virtual Reality.
car

Conclusion

This is an unique opportunity to claim the camera market with a product that can clearly differentiate itself for any offering in the marker making it the perfect driver not only for profit from sales but also from unlimited service opportunities from notifications for $0.99 a month to storage and other complex solutions based on machine learning and pattern recognition.

Adding suction cups to a Dropcam

After some emails on the neighborhood alerting about people stealing packages from the porch I decided to install an outside facing Dropcam.

I didn’t want to put it outside so my only option was to see through the window. That’s when it occurred to me that I could add some suction cups so I can be free to remove it anytime I want without any problem.

IMG_3128.JPG
Continue reading

Kid Proof Smart LEDStrip with SparkCore, Adafruit NeoPixels and SmartThings

I love to get my son excited every time I make or hack something so this time I created a SmartThings controlled Adafruit NeoPixels LED Strip for his bed. The plan was simple, LED Strip that can change colors and also be controlled by a simple and resistant button.

original

I will try to make it as simple as I can but please leave a comment if think that anything needs clarification.

Continue reading

Jawbone and SmartThings to the rescue (from an awkward date)

In an era where online dating increases every day, awkward dating is part of the picture. It often happens that you meet someone online and the first date is also the first time you interact with her/him in person.
Let’s be honest, your online profile is a way of marketing yourself, it usually reflects things you are proud of, leaving less pleasant aspect of your personality for the other to discover.
Matches are made in heaven but for those made in hell, now, you can use SmartThings and Jawbone.
jawbone-up-8The process is really simple, Continue reading

Bartendro, SmartThings and let the party begin

rsz_photo_15Bartendro works on a Raspberry Pi which hosts a little Web app that triggers the pumps. The challenge here was to be able to send commands from the SmartThings cloud to the little computer. For this demo, I created a Python script that queries a DB hosted outside the Bartendro that triggers the pumps if a new entry is found. I also created a Device Type on SmartThings, that adds an entry, with the necessary params, based on the drink selection. 


Before I release it to the SmartThings community I want to be able to send a JSON file to SmartThings with the list of drinks available for the bottles I have connected to the device. This will allow the device type to know which drinks to offer to the user. Also I’m thinking of using node.js for direct communication instead of queuing constantly a DB. Stay tuned!

Quick demo of NFC RFID reader working with SmartThings

This is a quick demo of my new project “RFID reader and SmartThings”.
The idea is that you can read RFID cards and record the ids that later you can use to trigger actions, for example, open the front door.

Hope you like the idea. Please leave comments on how do you think I can improve or use this project.

For a full step by step tutorial please click here

Thanks for watching!