Refer to video and the project page for more details.

Smart Cap is an assistant for visually impaired which narrates the description of the scene.

Why:

There are about 285 million visually impaired people in the world. They are not able to experience the world the way we do. Smart cap aims to provide this missing experience for them. The system uses state of the art deep learning techniques from Microsoft Cognitive Services for image classification and tagging.

What:

The smart cap aims bring the beautiful world as a narrative to the visually impaired. The narrative is generated by converting the scenes in front of them to text which describes the important objects in the scene. Examples of text include ‘A group of people playing a game of football’, ‘yellow truck parked next to the car’, a bowl of salad kept on the table’. For the first prototype of the system, one line along with some keywords are played as an audio to the users but in the later versions, a detailed description would be added as the feature.

How:

The architecture of the system includes Amazon Echo, Dragonboard 410c/Raspberry Pi and online computer vision API’s.

A webcam which is retrofitted into a regular cap is connected to the Dragonboard/Raspberry Pi. The code given here runs on Dragonboard 410c/Raspberry Pi. The function of the code is to capture the image from the webcam and send it to Microsoft API’s for recognition task. The response is then inserted to DynamoDB. When the user asks Alexa to describe the scene, the Alexa Skills Kit triggers Amazon Lambda function to fetch the data from the database (DynamoDB). The correct text is the played as an audio on the Alexa device.

The project was awarded by Qualcomm and Amazon.

Github: https://github.com/TusharChugh/SmartCap

To work towards the vision, we made robot Andy intelligent by enabling it to understand the speech commands for doing simple table top manipulation tasks. The project uses Alexa (Amazon Echo) for speech recognition.

Collaborators: William Seto, Bikramjot Hanzra, Shiyu Dong, Tae-Hyung Kim
Mentors: Jeah Oh (NREC), Katharina Muelling(NREC)

So, we though to decouple these scenarios from laptop so that people can access the same information without changing the tabs on their laptops. Here the idea of designing orientron came into picture.

We created a prototype of a device which a user can keep on the desk and subscribe to 4 different services providing the information selected by the user from phone/store app. Initially, we developed  for four application: cricket score, stock updates, weather updates and horoscope. The device is not having any button and the user can just rotate the device to change the on the screen. The vision is to build orientron APIs through which any user can subscribe to four services (each for a different orientation) from a list of many services, such as facebook, cricket score, weather update, horoscope etc.

Technology:

1. E-paper display: The device consumes power only when the content of the screen is refreshed. So, the device is highly power efficient

2. Bluetooth Connectivity: The device has built-in bluetooth and it can be paired with any device with bluetooth 2.0.

3. Windows Universal App: The device can connect to Windows Phone and Windows Store app from where user can select select 4 apps for 4 different orientation. We also started developing API’s so that other developers can also build different application for the device.

4. Accelerometer: To detect different orientations, and taps to refresh the content.

Collaborators: Rolly Seth

Timeline: July 2015

With the aim to build the next generation prototypes, devices, and services, this specially-designed physical space provides our employees the tools, technology, equipment, and the environment required to experiment with their ideas and interests.

In addition to this, we created a community of  more than 600 employees interested in Maker activities. Took workshops on topics and technologies related to  MS Band, Arduino, Raspberry Pi, IoT with Azure. build your own wearable including designing the PCB’s. The workshops and sessions were delivered in Hyderabad and Bangalore campus.

Team of 2 employees right from creating strategy to execution

Timeline:  Jan’15-July’15

Lync status: Available

Lync status: Do Not Disturb

Lync status: Away

Lync status: Busy

This is my latest fun project which I worked on in collaboration with Rolly Seth.

The device reflects the status of ‘Skype for Business’ (which is an enterprise equivalent of Skype and comes with the office suit of Microsoft). The main goal behind designing this device is to use this agent to increase the productivity of the employees in the organization. Having these device on the top of the cubicles displaying their status which can be easily interpreted by others, can help the employees to have control over their time by reducing interruptions from others . So, now employees can work in an open and collaborative environment and can have a control over their time (by choosing do not disturb/busy status) on the ‘skype for business’ application.

Technical details:

It has a microcontroller inside the devices which takes data serially from the laptop and then actuates the RGB LED on board. FTDI chip is used between laptop and the microcontroller (USB-serial).

On the laptop, a Windows background service runs in the background and constantly pulls that status of the user using ‘Skype for Business’ API’s.

The outer casing has been designed and 3D printed in view that it will give the device a cool form factor for them to use.

On my last night at Cambridge, I wanted to design something using the laser cutters in the lab and useless machine was the first  that popped-up in my mind. I stayed overnight in the lab to create this epic shit.

Wizitank: Top View

Wizitank: Side view

Our team ‘Wizminds’ presented ‘Wizitank’ at Microsoft Imagine Cup Poland. The vehicle is designed to handle low intensity conflicts and to to navigate through rugged terrains. It operated in two modes, autonomous and manual.