{"title": "Entrainment of Silicon Central Pattern Generators for Legged Locomotory Control", "book": "Advances in Neural Information Processing Systems", "page_first": 1043, "page_last": 1050, "abstract": "", "full_text": "Entrainment of Silicon Central Pattern Generators \n\nfor Legged Locomotory Control \n\n \n\nFrancesco Tenore1, Ralph Etienne-Cummings1,2, M. Anthony Lewis3 \n\n1Dept. of Electrical & Computer Eng., Johns Hopkins University, Baltimore, MD 21218 \n\n2Institute of Systems Research, University of Maryland, College Park, MD 20742 \n\n3Iguana Robotics, Inc., P.O. Box 625, Urbana, IL 61803 \n\n{fra, retienne}@jhu.edu, tlewis@iguana-robotics.com \n\n \n\n \n\nAbstract \n\nWe have constructed a second generation CPG chip capable of generating the necessary \ntiming  to  control  the  leg  of  a  walking  machine.  We  demonstrate  improvements  over  a \nprevious  chip  by  moving  toward  a  significantly  more  versatile  device.  This  includes  a \nlarger  number  of  silicon  neurons,  more  sophisticated  neurons  including  voltage \ndependent  charging  and  relative  and  absolute  refractory  periods,  and  enhanced \nprogrammability of neural networks. This chip builds on the basic results achieved on a \nprevious  chip  and  expands  its  versatility  to  get  closer  to  a  self-contained  locomotion \ncontroller for walking robots. \n \n1 \n \n\nIntroduction \n\nLegged  locomotion  is  a  system  level  behavior  that  engages  most  senses  and \nactivates  most  muscles  in  the  human  body.  Understanding  of  biological  systems  is \nexceedingly  difficult  and  usually  defies  any  unifying  analysis.  Walking  behavior  is  no \nexception. Theories of walking are likely incomplete, often in ways that are invisible to \nthe  scientist  studying  these  behavior  in  animal  or  human  systems.  Biological  systems \noften  fill  in  gaps  and  details.  One  way  of  exposing  our  incomplete  understanding  is \nthrough  the  process  of  synthesis.  In  this  paper  we  report  on  continued  progress  in \nbuilding  the  basic  elements  of  a  motor  pattern  generator  sufficient  to  control  a  legged \nrobot. The focus of this paper is on a 2nd generation chip, that incorporates new features \nwhich we feel important for legged locomotion. \n\nAn essential element of most locomotory systems is the Central Patter Generator \n(CPG). The CPG is a set of neural circuits found in the spinal cord, arranged to produce \noscillatory periodic waveforms that activate muscles in a coordinated manner. They are \nneuron  primitives  that  are  used  in  most  periodic  biological  systems  such  as  the \nrespiratory,  the  digestive  and  the  locomotory  systems.  In  this  last  one,  CPGs  are \nconstructed  using  neurons  coupled  together  to  produce  phasic  relationships  required  to \nachieve coordinated gait-type movements. \n\nThe  CPG  is  more  than  a  clock,  or  even  a  network  of  oscillators.    Phenomena \nsuch as reflex reversal [7] can only be understood in terms of a system that has at least \none additional state variable over sensory information alone.  The CPG or similar circuits \nis certainly involved in modulation of sensory information from the periphery [5] and is \nof  primary  importance  in  providing  phase  information  to  the  cerebellum.  This \ninformation is necessary for coordination of the brain and the spinal cord [6]. \n\nCurrently,  there  are  two  extremes  in  using  CPGs  for  control  of  mechanical \ndevices. The first is to be as faithful to the biological as possible, and then to discover \nhow biological systems can assist in the control of complex machines. This approach is \nsimilar to that of Rasche et al. [1], based on the Hodgkin-Huxley model [3], and the one \n\n\fimplemented by Simoni and DeWeerth [2], based on the Morris-Lecar model [4]. These \nion-channel based models imply a very large parameter space, making it difficult to work \nwith in silicon, yet inviting direct comparison with biological counterparts. \n\nOur approach is to start in the other direction. A system of minimal complexity \nwas  built  [8,9]  and  then  the  question  was  asked  of  what  additional  features  should  be \nadded to this minimal system to enable a behavior that is missing in the previous design. \nThus, the two approaches start from different philosophical grounds, but will, hopefully, \nconverge on similar solutions.  \n\nThe  motivation  behind  choosing  a  self-contained  silicon  system  rather  than  a \nsoftware implementation is that the former will use less power and be more compact and \nmore amenable to the control of a power-autonomous robot. \n\nPreviously,  a  minimal  system  chip  was  built  using  integrate-and-fire  neurons \ncontrolling a rudimentary robot [8, 9].  The chip described in this paper is an evolution of \nthat one. Its main differences with its previous version are the following. The previous \nchip  contained  2  spiking  motoneurons  and  2  pacemaker  neurons,  whereas  the  current \nchip contains 10 neurons of either type. More importantly, all the synapse weights (22 per \nneuron) are on-chip and can be used to make the synapse excitatory or inhibitory, while \nthe previous version weighted the synapse signals outside the chip. The current chip also \nhas 10 feedback synapses, making all the neurons interconnected. Moreover, the current \nchip has the capability of receiving and weighting up to 8 external inputs (instead of 2), \nsuch as sensory feedback signals, to allow better control of the CPG. The possibility of \nbetter  tuning  the  pacemaker  and  spiking  motoneurons  created  by  the  chip  is  achieved \nthrough direct modulation of the pulse width, of the absolute or relative refractory period \nand of the discharge strength of each neuron. Finally, the charging and discharging of the \nneurons\u2019  membrane  capacitance  is  an  exponential  function  of  time,  as  opposed  to  the \nlinear function that the previous chip exhibited. This allows for better coupling between \nCPGs (unpublished observation). \n\nIn  this  paper,  after  explaining  the  architecture  of  the  chip  and  how  simple \nnetworks can be created, a robotic application will be described. The paper will show that \nentrainment  of  multiple  CPGs  can  be  achieved  by  using  direct  coupling.  Analysis  and \nexperiments demonstrating entrainment between multiple CPGs using direct coupling are \npresented.  Finally,  the  oscillatory  patterns  used  to  control  a  single-legged  robot  are \nimplemented in this chip. \n\nThe CPG emulator chip was fabricated in silicon using a 0.5 m m CMOS process. \nThe  chip  was  designed  to  provide  plausible  electronic  counterparts  of  biological \nelements,  such  as  neurons,  synapses,  cell  membranes,  axons,  and  axon  hillocks,  for \ncontrolling motor systems. The chip also contains digital memories that can be used with \nsynapses  to  modify  weights  or  to  modulate  the  membrane  conductance.  Through  these \ncomponents,  it  is  possible  to  construct  non-linear  oscillators,  which  are  based  on  the \ncentral pattern generators of biological organisms.  \nThe chip\u2019s architecture can be seen in figure 1. It is made up of 10 fully interconnected \n\u201cneurons\u201d  and  22  \u201csynapses\u201d  per  neuron.  Communication  with  a  particular \nneuron/synapse  pair  occurs  through  the  address  register,  made  up  of  the  neuron/row \nregister and the synapse/column register. Finally, a weight/data register allows a tunable \namount of current to flow onto or away from the \u201cneurons\u2019 axons.\u201d \n\nFigure 2 shows a detailed view of a single neuron. As can be seen, all neurons \n\nare integrate-and-fire type neurons, in which the current that flows on the axon charges  \n\n \nArchitecture \n\n2 \n \n\n\fWeight Value\n\nSynapse/Column Select\n\nFeedback\n\n...\n\n \n\n \n\nt\nc\ne\nl\ne\nS\nw\no\nR\nn\no\nr\nu\ne\nN\n\n/\n\nNeuron 1\n\nNeuron 2\n\nNeuron 3\n\nNeuron 4\n\nNeuron 5\n\nNeuron 6\n\nNeuron 7\n\nNeuron 8\n\nNeuron 9\n\n \n\nVout\n1\n\nVout\n\n2\n\nVout\n\n3\n\nVout\n\n4\n\nVout\n5\n\nVout\n\nVout\n\n6\n\n7\n\nVout\n8\n\nVout\n9\n\nNeuron 10\n\nVout\n\n10 \n\nFigure  1.  Top.  Chip  micrograph,  3.3x2.1  mm2.  The  22  synapses  per  neuron  (vertical \nlines) are distinguishable. Bottom. System block diagram. \n \nup the membrane capacitor, Cmem. When the voltage across the capacitor reaches a certain \nthreshold, Vthresh, the hysteretic comparator output goes high. The output of the  \ncomparator does not change if the discharge and refractory period controls are disabled. \nNormally, however, the discharge controller is active and its function is to decrease the \nvoltage  on  the  membrane  capacitance  until  it  drops  below  the  hysteretic  comparator\u2019s \nlower threshold. The comparator output then goes low, the discharge is halted, and the \ncapacitor can charge up again, thereby making the process start anew. \n\nmem\n\nThe i-th neuron can be modeled through the following set of  equations: \ndV\ni\ndt\n\nIW\nik\n\nIW\nij\n\nIS\ni\n\nIS\ni\n\ndis\n\n= (cid:229)\n\n \n\nrefrac\n\n \n\n+\n\nj\n\nj\n\nk\n\nk\n\nmem\n\nC\n\ni\n\n \n\n(1) \n\n(2) \n\ntSi\n(\n\n+\n\ndt\n\n)\n\n=\n\n1\n0\n\n   \n\nif\nif\n\n   \n\n(cid:217)=\n(\n1)(\ntS\ni\n(cid:217)=\ntS\n0\n(\n)(\ni\n\nmem\n\nmem\n\nV\ni\nV\ni\n\n>\n>\n\nV\nT\nV\nT\n\n)\n)\n\n+\n\nmem\n\nmem\n\n(\nV\ni\n(\nV\ni\n\n+\n\n>\n>\n\nV\nT\nV\nT\n\n)\n)\n\n \n\n \n+ and VT\n- are respectively \nmem is the membrane capacitance of the i-th neuron, VT\nwhere Ci\nmem  is  the  voltage  on  the \nthe  high  and  low  thresholds  of  the  hysteretic  comparator,    Vi\ncapacitor,  Si(t)  is  the  state  of  the  hysteretic  comparator  at  time  t,  W+\nij  is  the  excitatory \nweight  on  the  j-th  excitatory  synapse  of  the  i-th  neuron  and  similarly  W -\nik  is  the \ninhibitory  weight  on  the  k-th  inhibitory  synapse  of  the  i-th  neuron.  The  discharge  and \nrefractory currents, Idis and Irefrac correspond to the discharge and refractory period rates, \nrespectively. \n\n-\n-\n-\n(cid:229)\n-\n(cid:238)\n(cid:237)\n(cid:236)\n-\n-\n(cid:218)\n(cid:218)\n\f \n\nNeuron 1\n\nAn\n\n1\n\nAn\n\n4 Dig\n\n1\n\nDig\n\n4\n\nVout1\n\nVout10\n\n...\n\n...\n\n...\n\nAxon Hillock\n\n(Hysteretic Comparator)\n\nInternal bias\n\nAnalog Inputs\n\nDigital Inputs\n\nFB Signals\n\nWeight (Exc)\n\nWeight (Exc)\n\nWeight (Exc)\n\nWeight (Exc)\n\nWeight (Inh)\n\nWeight (Inh)\n\nWeight (Inh)\n\nWeight (Inh)\n\nC\n\nmem\n\nAxon\n\nInternal bias\n\nAnalog Inputs\n\nDigital Inputs\n\nFB Signals\n\nV\n\nthresh\n\ndisI\n\n...\n\n...\n\nAn\n\n1\n\nAn\n\n4\n\nDig\n\n1\n\nDig\n\n4\n\nVout1\n\nI\n\nrefrac\n\n...\n\nVout10\n\nDischarge\n\nRefrac Control\n\nPW Control\n\nVout1\n\n \nFigure  2.  Block  diagram  of  a  single  neuron.  The  neuron  output  is  fed  back  to  all  the \nneurons including itself (Vout1 is also a feedback signal). \n\n \nThe speed with which the comparator changes state depends on the amount of \ncurrent that the weight, or weights, sets on or remove from the \u201caxon\u201d. The weights are \nset through 8-bit digital-to-analog converters (DACs) and stored in static random access \nmemory  (SRAM)  cells.  A  ninth  bit  selects  the  type  of  weight,  either  excitatory  or \ninhibitory.  Finally,  the  three  blocks  that  depend  on  the  comparator  output,  work  as \nfollows. A weight can be set on any one of these three blocks, just as was done for the \nsynapses. This allows modulation of the discharge strength, of the refractory period, and \nof the pulse width. The refractory period control element prevents current from charging \nup the capacitor for as long as it is active. It can be both relative and absolute, depending \non its weight. The pulse-width block allows independent control of the output duty cycle \nby modifying the amount of time the output is high. As can be seen in figure 2, the output \nfrom the PW control block is both the neuron output and the feedback signal to all the \nneurons, including itself (self-feedback). The chip is thus fully interconnected. \n\nFrom figure 2, four types of synapses can be identified. The first is the internal \nbias synapse, which allows current to flow onto or away from the membrane capacitor, \ndepending on the type of bias it has, without requiring signals from inside the chip. The \nanalog and digital synapses require the presence of an external analog or digital voltage \nto allow current to flow on the capacitor. The feedback synapses are also internal to the \nchip and allow the neurons to influence each other by modulating the charge-up of the \nmembrane capacitors they are acting upon. This means that one of these synapses is of \nself-feedback for a particular neuron. These synapses are considered to be dual mode, in \nthat they can both excite or inhibit. The 3 final synapses are used to control the discharge \nstrength, the refractory period, and the pulse width. \n\nIt  is  thus  possible  to  attain  two  types  of  waveforms  at  each  neuron  output, \ndepending on the current charging the capacitor. If the current charges up and discharges \nthe capacitor very quickly, the output is similar to that of a motor neuron. If the current \ncharges  and  discharges  the  capacitor  slowly,  then  the  output  is  that  of  a  pacemaker \nenvelope neuron, which makes up the CPG. \n \n3 \n \n\nNetworks \n\nTwo simple networks are described in this section using this chip to understand \n\nthe how the chip operates. The first example is shown in figure 3. A pacemaker neuron  \n\n\fbias\nsynapse\n\nbias\nsynapse\n\nVthresh\n\n+I\n\nCmem\n\ndisI\n\nDischarge\n\nfeedback\nsynapse\n  (exc)\n\nVout\n\nPW Control\n\nfeedback\nsynapse\n\nVthresh\n\n+I\n\nCmem\n\ndisI\n\nDischarge\n\nPW Control\n\nVout\n\n \nFigure 3. An envelope neuron exciting a motor neuron. The output waveforms are 180\u00ba \nout-of-phase. \n \n\n \n\n \n\n \nFigure  4.  Master  slave  relationship.  When  the  master  spikes,  the  membrane  potential \nincreases for the duration of the spike. \n \ncontrols the spiking of a motor neuron such that the spiking only occurs if the envelope is \nhigh.  This  is  done  using  the  internal  biasing  synapse  to  charge  up  the  membrane \ncapacitance of the envelope neuron and the feedback synapse coming from the envelope \nneuron  to  charge  up  the  capacitor  of  the  motor  neuron.  Similarly,  the  envelope  neuron \ncan inhibit the spiking which would otherwise occur at a constant rate through the bias \nsynapse.  Note  that  the  bias  synapse  can  either  be  the  internally  generated,  as  the  one \nshown in figure 3, or it can be the one of the external analog or digital synapse seen in \nfigure 2.  \n\nA second example, shown in figure 4, depicts the effects of a single spike on an \nenvelope neuron. Depending on where the spike occurs with respect to the slave envelope \nneuron, it will either accelerate the charge-up or decelerate the discharge. In this example, \nthe  spike  occurred  during  the  membrane  potential\u2019s  discharge  phase.  The  membrane \npotential\u2019s output voltage is shown within the slave output waveform. The two horizontal \nlines  that  delimit  it  represent  the  hysteretic  comparator\u2019s  threshold  voltages.  Thus,  the \nslave  stays  high  for  a  longer  period  of  time,  thus  decreasing  its  normal  frequency  of \noscillation. It is therefore possible to entrain the slave oscillator to the frequency of the \nmaster. This can be done either by increasing the duration of the master spike, increasing  \n\n\fMaster oscillator\n\nSpike Entrainer\n\nSlave oscillator\n\nbias\nsynapse\n\n   Spike\nDischarger\n\nbias\nsynapse\n\nFigure 5. CPG entrainment. \n \n\nMaster\n\nSpike Entrainer\n\nSpike discharger\nSlave\n\n \n\n \n\nFigure 6. Phase delay between master envelope and spike entrainer.  \n \nthe feedback weight with which the master controls the slave, or simply by increasing the \nspike frequency. For example, in this latter case, if the master frequency is higher than \nthe slave\u2019s, then the spike will accelerate the slave such that it reaches the same period. \n \n4 \n\nAnalysis of pulse coupling \n \nTo show that it is possible to entrain two oscillators to have the same frequency \nbut  alter  the  phase  at  will,  such  that  any  phase  between  the  two  waveforms  can  be \nachieved,  it  is  necessary  to  use  a  configuration  similar  to  the  one  described  in  the \nprevious section. A master and slave oscillator with different frequencies and both with \napproximately 50% duty cycle are set up as shown in figure 5. Another neuron is used to \ngenerate  a  single  spike  during  the  master\u2019s  pulse  width  called  the  entrainer  spike.  It  is \nevoked by the input from the master and has the same frequency, but its phase depends \non the strength of the feedback synapse between these two cells. The spike\u2019s discharge \noccurs very slowly, but to ensure that no residual charge is left on the capacitor, a fourth \nneuron, 180\u00ba out-of-phase with the master, is used. When this neuron is high, it sends a \nstrong inhibition signal to the spike, thereby resetting it. At this point, the spike can be \nused  for  synchronizing  the  slave  oscillator.  As  described  previously,  if  the  slave \noscillator\u2019s frequency is lower than the master\u2019s (and therefore that of the spike\u2019s), the \nspike\u2019s  effect  is  to  accelerate  until  the  two  are  synchronized.  This  allows  for  two \npacemaker neurons to be out-of-phase by an arbitrary angle. This is shown in figure 6, \nwhere the coupling weight between master and slave was systematically altered and the \nresulting phase variation was recorded. To fine tune the slave oscillator\u2019s desired phase \ndifference,  once  the  spike  master  has  been  set,  it  is  necessary  to  tune  the  feedback \nstrength between the spike and the slave oscillator. A stronger feedback will allow the  \n\n\f1\n+\nN\n\ne\ns\na\nh\nPhase (N+1)\nP\n\n1\n\n0.9\n\n0.8\n\n0.7\n\n0.6\n\n0.5\n\n0.4\n\n0.3\n\n0.2\n\n0.1\n\n0\n\n0\n\nMap Function (4.4 ms pulse width)\n\nSlope = -1\n\nSlope of |f(x)|< 1\n\n0.2\n\n0.4\n\n0.6\n\n0.8\n\n1\n\nPhase\nN\n\n \n\n       Experiment \n\n5 \n \n\n \n\nFigure 7. Map function illustrating the coupling behavior between two neurons.  \n \n\ntwo  signals  to  happen  virtually  at  the  same  time,  a  weaker  weight  will  cause \nsome delay between the two. Lewis and Bekey show that adaptation of time is critical to \ncontrolling walking in a robot [10]. \n\nFinally,  figure  7  shows  a  map  function  obtained  using  a  4.4  ms  spike  pulse \nwidth. A map function depicts the effect of a spike on a pacemaker neuron at all possible \nphases.  The  curve  shows  a  slope  smaller  1  (in  absolute  value)  in  the  transition  region, \nwhich implies that the system is asymptotically stable [9]. \n\nTo build on all the results achieved, the oscillatory patterns necessary to control \na  single-legged  robot  were  synthesized.  Figure  7  shows  the  waveforms  generated  to \ncontrol  a  hip\u2019s  flexor  and  extensor  muscles  and  ipsilateral  knee\u2019s  flexor  and  extensor. \nThese  waveforms  were  generated  using  all  10  available  neurons  with  the  procedures \ndescribed  previously.  The  hip  flexor  and  extensor  are  180\u00ba  out-of-phase  to  each  other. \nThe  left  knee  extensor  is  slightly  out-of-phase  with  its  respective  hip  muscle  but  the \nwidth of the waveform\u2019s pulse is shorter than that of the hip extensor. As can be seen, the \nknee flexor has two bumps, where the purpose of the first bump is to stabilize the knee \nwhen the foot hits the substrate. The waveforms depicted are necessary to drive a robotic \nleg with a standard walking gait. Different gaits will have waveforms with different phase \nrelationships.  However,  the  results  shown  in  the  previous  sections  show  that  these \nwaveforms,  through  simple  variations  of  the  timing  parameters  described,  can  be \ngenerated with ease. \n \n6 \n \n\nConclusions \n\nThe waveforms needed to control a robotic leg can be generated using a silicon \nchip  described  in  this  paper.  The  phase  differences  between  the  waveforms,  however, \nchange depending on the type of gait that one wants to implement in a robot. The results \nobtained  show  that  any  phase  difference  between  two  or  more  waveforms  can  be \nachieved, thus making any gait effectively achievable. Furthermore, the map function that \n\n\fresulted  from  on-chip  measurements  showed  that  the  chip  has  the  capability  of \nasymptotic coupling stability. \n\n \n\n \n\nFigure 8. Waveforms generated to control a robotic leg. \n \nReferences \n[1].  C.  Rasche,  R.  Douglas,  M.  Mahowald,  \u201cCharacterization  of  a  pyramidal  silicon \nneuron,\u201d Neuromorphic Systems: Engineering silicon from neurobiology, L. S. Smith and \nA. Hamilton, eds, World Scientific, 1st edition, 1998. \n[2].  M.  Simoni,  S.  DeWeerth,  \u201cAdaptation  in  an  aVLSI  model  of  a  neuron.\u201d  IEEE \nTransactions on circuits and systems II: Analog and digital signal processing. 46(7):967-\n970, 1999. \n [3]. A.L. Hodgkin, A.F. Huxley. \u201cA quantitative description of ion currents and its \napplications to conduction and excitation in nerve membranes,\u201d Journal of Physiology \n(Lond.), 117:500-544, 1952. \n[4].  C.  Morris,  H.  Lecar,  \u201cVoltage  oscillations  in  the  barnacle  giant  muscle  fiber,\u201d \nBiophysics J., vol. 35, pp. 193-213, 1981.  \n[5]. Y.I. Arshavsky, I. M. Gelfand, and G. N. Orlovsky, \u201cThe cerebellum and control of \nrhythmic movements,\u201d TINS, vol. 6, pp. 417-422, 1983. \n[6]. A.H. Cohen, D.L. Boothe, \u201cSensorimotor interactions during locomotion: principles \nderived  from  biological  systems,\u201d  Autonomous  robots,  special  issue  on  biomorphic \nrobots, M.A. Lewis and M.A. Arbib, (Eds). Vol. 7, pp. 225-238, 1999. \n[7]. H. Forssberg, S. Grillner, S. Rossignol, \u201cPhase dependent reflex during walking in \nchronic spinal cats,\u201d Brain research, vol. 85, pp. 103-7, 1975. \n [8].  M.A.  Lewis,  R.  Etienne-Cummings,  A.H.  Cohen,  M.  Hartmann,  \u201cToward \nbiomorphic  control  using  custom  aVLSI  chips\u201d,  Proceedings  of  the  International \nconference on robotics and automation, San Francisco, CA, 2000. \n[9].  M. A. Lewis, R. Etienne-Cummings, M. J. Hartmann, A. H. Cohen, Z. R. Xu, \u201cAn in \nsilico  central  pattern  generator:  silicon  oscillator,  coupling,  entrainment,  and  physical \ncomputation\u201d, Biological Cybernetics, 88, 2, 2003, pp. 137-151. \n[10]. M. Anthony Lewis and George A. Bekey (2002), Gait Adaptation in a Quadruped \nrobot, Autonomous Robots, 12(3) 301-312. \n\n\f", "award": [], "sourceid": 2533, "authors": [{"given_name": "Francesco", "family_name": "Tenore", "institution": null}, {"given_name": "Ralph", "family_name": "Etienne-Cummings", "institution": null}, {"given_name": "M.", "family_name": "Lewis", "institution": null}]}