Search

Register

Thanks for signing-up! You will receive an email with further instructions to activate your account!

username already taken
You already have an account? .

Login

Forgot password?
Don't have an account yet? Register now for using the UI-Designer and the forum or creating projects.

Controller for Rapiro (A Raspberry Pi Powered Robot)

Project Information

Attached UI-Designer Configs
Rapiro Controller
A controller for the Raspberry Pi-brained robot. I am going to pack movement control as well as various functions includes camera related, voice recognition, text to speech etc into a single UI. The resolution setting is optimized for Xperia Z.

Description

Rapiro (http://www.rapiro.com/) is a Raspberry Pi based robot kit from Japan. The project succeed both in Kickstarter funding and another Japanese domestic funding, and will be released for sale this weekend. I have just got mine delivered about a week ago and am trying with him these days.

The robot is moved by 12 servos. These servos are driven by the Rapiro board, an Arduino compatible board. Then as its brain, a Raspberry Pi is connected to the Rapiro board and all the movements, sensor processing of the board can be handled on the Pi. Including the camera, speaker, microphone etc attached to the Pi, this robot has huge possibility as an intelligent agent. (Of course it is not as sophisticated as the multi-thousand dollar bots of the big labs)

What I'm trying to do is to pack essential interaction with the robot into a single UI. The function list includes:
  • Servo related:
    - Power supply to the servos <completed>
    - Walking direction <completed>
    - Arm angle and grip <completed>
    - Head and waist angle <completed>
    - Color of the eyes (no practical meaning, but it can be changed, so why not)
    - Some pre-programmed movement (there are 5 of them coming with the robot, very cute)<completed>

  • Sound related:<mostly completed>
    - Voice recognition (google api)
    - Intelligent agent (wolfram alpha api)
    - Text to speech (espeak/google api)
    - Specific functions (read news headlines, tell current weather etc)

  • Camera related:
    - Take still picture <completed>
    - Stream video like (1 fps) time-lapse as cockpit view <completed>
    - Set time-lapse picture taking
    - Face recognition

And here is the TODO list.
- Take care of the camera vs Pi power problem
- Configure espeak to a better voice <completed>
- Implement eye color change function
- Add more function to voice recognition/voice based control
- Embed camera picture from the robot into the UI <completed>

Any suggestion, advice to the functions/UI are welcomed! I'll update a reference list of the tutorials/blogs helped me to implement the functions.

——————————–20140304 update:——————————–
A list of libraries needed for the server script.
  • - wolframalpha(python library) and you need to obtain an ID to use the API
    - feedparser(python library)
    - minicom
    - espeak
    - mjpeg_streamer(cannot be displayed within NetIO web view, so this is for other browsers)
    - apache2
I also added a command to the Rapiro's Arduino script, #Z for an instant servo power off.
Voice recognition and image recognition functions will be included in the next step.

Server script ‘netio_server.py’
import asyncore
import socket
import select
import RPi.GPIO as GPIO
import os
import wolframalpha
import sys
import re
import feedparser
# Get a free API key here http://products.wolframalpha.com/api/
app_id=‘YOUR_WOLFRAMALPHA_ID’
client = wolframalpha.Client(app_id)
def QueryWolframalpha(query):
    res = client.query(query)
    if len(res.pods) > 0:
        texts = ""
        pod = res.pods[1]
        if pod.text:
            texts = pod.text
        else:
            texts = "I have no answer for that"
        # to skip ascii character in case of error
        texts = texts.encode('ascii', 'ignore')
        return texts
    else:
        return "Sorry, I am not sure."
def PrepareTextForEspeak(input):
    output = re.sub(re.compile("[!-/:-@[-`{-~]"),'',input)
    #return output.split('\n')[0]
    return output
def GetNewsHeadlines(URL):
    return feedparser.parse(URL)
def FormatNewsFeed(text):
    res = text.feed.title
    for item in text.entries:
        res = res + '. ' + item.title
    return res
def Speak(sentence):
    command_line = 'espeak -s120 -k18 -a200 -v female5 \"%s\"' % sentence
    os.system(command_line)
def StartCaptureTimeLapse():
    command_line = 'raspistill -tl 1000 -t 600000 -w 640 -h 480 -q 5 -o /var/www/capture.jpg --nopreview -th 0:0:0 &'
    os.system(command_line)
    #command_line = 'LD_LIBRARY_PATH=/usr/local/lib/ mjpg_streamer -i \"input_file.so -f /var/www/ -n capture.jpg\" -o \"output_http.so -w /usr/local/www\" &'
    #os.system(command_line)
    print command_line
def StopCaptureTimeLapse():
    #command_line = 'kill $(pgrep mjpg_streamer)'
    #os.system(command_line)
    command_line = 'kill $(pgrep raspistill)'
    os.system(command_line)
    print command_line
#Initial GPIO-setup
GPIO.setwarnings(False)
GPIO.cleanup()
# to use Raspberry Pi board pin numbers
GPIO.setmode(GPIO.BCM)
# For LED1 we use pin 4 according BCM pin count 
# (see https://projects.drogon.net/raspberry-pi/wiringpi/pins/)
LED1 = 4
GPIO.setup(LED1, GPIO.OUT)
# For Switch input we use pin 17 according BCM pin count
SWITCH1 = 17
# set up GPIO input with pull-up control
#   (pull_up_down be PUD_OFF, PUD_UP or PUD_DOWN, default PUD_OFF)
GPIO.setup(SWITCH1, GPIO.IN, pull_up_down=GPIO.PUD_UP)
#camera capture mode
current_capture_state = 'capture off'
#servo power mode
current_power_state = 'on'
#arm angle
left_arm_angle = '180'
right_arm_angle = '0'
#shoulder angle
left_shoulder_angle = '50'
right_shoulder_angle = '130'
#head and waist
head_angle = '90'
waist_angle = '90'
class Client(asyncore.dispatcher_with_send):
    def __init__(self, socket=None, pollster=None):
        asyncore.dispatcher_with_send.__init__(self, socket)
        self.data = ''
        if pollster:
            self.pollster = pollster
            pollster.register(self, select.EPOLLIN)
    def handle_close(self):
        if self.pollster:
            self.pollster.unregister(self)
    def handle_read(self):
        receivedData = self.recv(8192)
        if not receivedData:
            self.close()
            return
        receivedData = self.data + receivedData
        while '\n' in receivedData:
            line, receivedData = receivedData.split('\n',1)
            self.handle_command(line)
        self.data = receivedData
    def handle_command(self, line):
        global current_capture_state
        global current_power_state
        global left_arm_angle
        global right_arm_angle
        global left_shoulder_angle
        global right_shoulder_angle
        global head_angle
        global waist_angle
        if line == 'Power On':
            self.send('on')
            current_power_state = 'on'
            command_line = 'echo \"#M0\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0'
            os.system(command_line)
            #Speak('Servo power on')
            print 'servo power on'
        elif line == 'Power Off':
            self.send('off')
            current_power_state = 'off'
            command_line = 'echo \"#Z\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0'
            os.system(command_line)
            #Speak('Servo power off')
            print 'servo power off'
        elif line == 'Get Status':
            self.send(current_power_state)
        elif line == 'Capture On':
            self.send('capture on')
            current_capture_state = 'capture on'
            StartCaptureTimeLapse()
            Speak('Capture on')
            print 'capture on'
        elif line == 'Capture Off':
            self.send('capture off')
            current_capture_state = 'capture off'
            StopCaptureTimeLapse()
            Speak('Capture off')
            print 'capture off'
        elif line == 'Get Capture Status':
            self.send(current_capture_state)
        elif line == 'Get LArm Status':
            self.send(left_arm_angle)
        elif line == 'Get RArm Status':
            self.send(right_arm_angle)
        elif len(line.strip().split(' ')) == 2 and line.strip().split(' ')[0] == 'SetLArmTo':
            self.send('unknown command')
            left_arm_angle = line.strip().split(' ')[1]
        elif len(line.strip().split(' ')) == 2 and line.strip().split(' ')[0] == 'SetRArmTo':
            self.send('unknown command')
            right_arm_angle = line.strip().split(' ')[1]
        elif line == 'Get LShld Status':
            self.send(left_shoulder_angle)
        elif line == 'Get RShld Status':
            self.send(right_shoulder_angle)
        elif len(line.strip().split(' ')) == 2 and line.strip().split(' ')[0] == 'SetLShldTo':
            self.send('unknown command')
            left_shoulder_angle = line.strip().split(' ')[1]
        elif len(line.strip().split(' ')) == 2 and line.strip().split(' ')[0] == 'SetRShldTo':
            self.send('unknown command')
            right_shoulder_angle = line.strip().split(' ')[1]
        elif line == 'Get Head Status':
            self.send(head_angle)
        elif line == 'Get Waist Status':
            self.send(waist_angle)
        elif len(line.strip().split(' ')) == 2 and line.strip().split(' ')[0] == 'SetHeadTo':
            self.send('unknown command')
            head_angle = line.strip().split(' ')[1]
        elif len(line.strip().split(' ')) == 2 and line.strip().split(' ')[0] == 'SetWaistTo':
            self.send('unknown command')
            waist_angle = line.strip().split(' ')[1]
        elif line == 'Grip Left':
            self.send('unknown command')
            command_line = 'echo \"#PS07A130T010\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0'
            os.system(command_line)
        elif line == 'Grip Right':
            self.send('unknown command')
            command_line = 'echo \"#PS04A050T010\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0'
            os.system(command_line)
        elif line == 'Release Left':
            self.send('unknown command')
            command_line = 'echo \"#PS07A090T010\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0'
            os.system(command_line)
        elif line == 'Release Right':
            self.send('unknown command')
            command_line = 'echo \"#PS04A090T010\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0'
            os.system(command_line)
        elif line == 'Move Head':
            self.send('unknown command')
            command_line = 'echo \"#PS00A%03dT010\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0' % int(head_angle)
            os.system(command_line)
            print "move head to %s" % head_angle
        elif line == 'Move Waist':
            self.send('unknown command')
            command_line = 'echo \"#PS01A%03dT010\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0' % int(waist_angle)
            os.system(command_line)
            print "move waist to %s" % waist_angle
        elif line == 'Move L Arm':
            self.send('unknown command')
            command_line = 'echo \"#PS06A%03dT010\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0' % int(left_shoulder_angle)
            os.system(command_line)
            command_line = 'echo \"#PS05A%03dT010\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0' % int(left_arm_angle)
            os.system(command_line)
            print "move left arm to %s shoulder to %s" % (left_arm_angle, left_shoulder_angle)
        elif line == 'Move R Arm':
            self.send('unknown command')
            command_line = 'echo \"#PS03A%03dT010\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0' % int(right_shoulder_angle)
            os.system(command_line)
            command_line = 'echo \"#PS02A%03dT010\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0' % int(right_arm_angle)
            os.system(command_line)
            print "move right arm to %s shoulder to %s" % (right_arm_angle, right_shoulder_angle)
        elif line == 'Move Forward':
            self.send('unknown command')
            command_line = 'echo \"#M1\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0'
            os.system(command_line)
            print 'Move Forward'
        elif line == 'Move Back':
            self.send('unknown command')
            command_line = 'echo \"#M2\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0'
            os.system(command_line)
            print 'Move Back'
        elif line == 'Move Left':
            self.send('unknown command')
            command_line = 'echo \"#M3\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0'
            os.system(command_line)
            print 'Move Left'
        elif line == 'Move Right':
            self.send('unknown command')
            command_line = 'echo \"#M4\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0'
            os.system(command_line)
            print 'Move Right'
        elif line == 'Move Stop':
            self.send('unknown command')
            command_line = 'echo \"#M0\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0'
            left_arm_angle = '180'
            right_arm_angle = '0'
            left_shoulder_angle = '50'
            right_shoulder_angle = '130'
            head_angle = '90'
            waist_angle = '90'
            os.system(command_line)
            print command_line
        elif line == 'Pose 5':
            self.send('unknown command')
            Speak('Activity number five')
            command_line = 'echo \"#M5\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0'
            os.system(command_line)
            print command_line
        elif line == 'Pose 6':
            self.send('unknown command')
            Speak('Activity number six')
            command_line = 'echo \"#M6\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0'
            os.system(command_line)
            print command_line
        elif line == 'Pose 7':
            self.send('unknown command')
            Speak('Activity number seven')
            command_line = 'echo \"#M7\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0'
            os.system(command_line)
            print command_line
        elif line == 'Pose 8':
            self.send('unknown command')
            Speak('Activity number eight')
            command_line = 'echo \"#M8\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0'
            os.system(command_line)
            print command_line
        elif line == 'Pose 9':
            self.send('unknown command')
            Speak('Activity number nine')
            command_line = 'echo \"#M9\" | sudo minicom -b 57600 -o -D /dev/ttyAMA0'
            os.system(command_line)
            print command_line
        elif line == 'Open Speaker':
            self.send('unknown command')
            Speak('Open speaker')
            command_line = 'aplay /home/pi/aquestalkpi/se9.wav'
            os.system(command_line)
            print command_line
        elif line == 'Take Picture':
            self.send('unknown command')
            Speak('Take a picture')
            command_line = 'raspistill -w 640 -h 480 -q 10 -o ./test.jpg --nopreview'
            os.system(command_line)
            print command_line
        elif line == 'Voice Recognition':
            self.send('unknown command')
            Speak('Start voice recognition')
        elif line == 'Read News':
            self.send('unknown command')
            Speak('Read news head lines')
            text = GetNewsHeadlines('http://feeds.bbci.co.uk/news/rss.xml')
            formatted_text = FormatNewsFeed(text)
            Speak(formatted_text)
        elif line == 'Tell Weather':
            self.send('unknown command')
            Speak('Check weather condition')
            query = 'current weather'
            answer = QueryWolframalpha(query)
            striped_answer = PrepareTextForEspeak(answer)
            print striped_answer
            Speak(striped_answer)
        else:
            Speak('Unknown command signal')
            self.send('unknown command')
       
class Server(asyncore.dispatcher):
    def __init__(self, listen_to, pollster):
        asyncore.dispatcher.__init__(self)
        self.pollster = pollster
        self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
        self.bind(listen_to)
        self.listen(5)
    def handle_accept(self):
        newSocket, address = self.accept()
        print "Connected from", address
        Client(newSocket,self.pollster)
def readwrite(obj, flags):
    try:
        if flags & select.EPOLLIN:
            obj.handle_read_event()
        if flags & select.EPOLLOUT:
            obj.handle_write_event()
        if flags & select.EPOLLPRI:
            obj.handle_expt_event()
        if flags & (select.EPOLLHUP | select.EPOLLERR | select.POLLNVAL):
            obj.handle_close()
    except socket.error, e:
        if e.args[0] not in asyncore._DISCONNECTED:
            obj.handle_error()
        else:
            obj.handle_close()
    except asyncore._reraised_exceptions:
        raise
    except:
        obj.handle_error()
class EPoll(object):
    def __init__(self):
        self.epoll = select.epoll()
        self.fdmap = {}
    def register(self, obj, flags):
        fd = obj.fileno()
        self.epoll.register(fd, flags)
        self.fdmap[fd] = obj
    def unregister(self, obj):
        fd = obj.fileno()
        del self.fdmap[fd]
        self.epoll.unregister(fd)
    def poll(self):
        evt = self.epoll.poll()
        for fd, flags in evt:
            yield self.fdmap[fd], flags
if __name__ == "__main__":
    pollster = EPoll()
    pollster.register(Server(("",5487),pollster), select.EPOLLIN)
    while True:
        evt = pollster.poll()
        for obj, flags in evt:
            readwrite(obj, flags)

Attachments