Dr. E.J. Chichilnisky: How the Brain Works, Curing Blindness & How to Navigate a Career Path

welcome to the huberman Lab podcast

where we discuss science and

science-based tools for everyday

[Music]

life I'm Andrew huberman and I'm a

professor of neurobiology and

Opthalmology at Stanford School of

Medicine my guest today is Dr EJ chelski

Dr EJ chelski is a professor of

neurosurgery Opthalmology and

Neuroscience at Stanford University he

is one of the world's leading

researchers trying to understand how we

see the world around us that is how

visual perception occurs and then

applying that information directly to

the design of neural prosthesis

literally robotic eyes that can allow

blind people to see once again today's

discussion is a very important one for

anyone who wants to understand how their

brain works indeed EJ spells out in very

clear terms exactly how the world around

us is encoded by the neurons the nerve

cells within our brain in order to

create these elaborate visual images

that we essentially see within our minds

and with that understanding he explains

how that can be applied to engineer

specific robotic Ai and machine learning

devices that can allow human brains not

only to see once again in the blind but

also to perceive things that typical

human brains can't and indeed for memory

to be enhanced and for cognition to be

enhanced this is the direction that

Neuroscience is going and in the course

of today's discussion we have the

opportunity to learn from the world

World expert in these topics where the

science is now and where it is headed

during today's discussion we also get

heavily into the topic of how to select

one's professional and personal path and

indeed you'll learn from Dr chelski that

he has a somewhat unusual path both into

science and through science so for those

of you that believe that everyone that's

highly accomplished in their career

always knew exactly what they wanted to

do at every stage you will soon learn

that that is absolutely not the case

with EJ he described

wandering through three different

graduate programs taking several years

off from school in order to dance Yes

you heard that correctly to dance and

how that wandering and indeed dancing

helped him decide exactly what he wanted

to do with his professional life and

exactly what specific problems to try

and Tackle in the realm of Neuroscience

and medicine it's a discussion that I'm

certain that everybody scientist or no

young or old can benefit from and can

apply the specific tools that EJ

describes in their own life and Pursuits

before we begin I'd like to emphasize

that this podcast is separate from my

teaching and research roles at Stanford

it is however part of my desire and

effort to bring zero cost to Consumer

information about science and science

related tools to the general public in

keeping with that theme I'd like to

thank the sponsors of today's podcast

our first sponsor is eight sleep eight

sleep makes Smart mattress covers with

cooling Heating and sleep tracking

capacity I've spoken many times before

on this other podcast about the fact

that sleep is the foundation of mental

health physical health health and

performance now one of the key aspects

to getting a great night's sleep is to

control the temperature of your sleeping

environment and that's because in order

to fall and stay deeply asleep your body

temperature actually has to drop by

about 1 to 3° and in order to wake up in

the morning feeling refreshed your body

temperature actually has to increase by

about 1 to 3 degrees eight sleep makes

it extremely easy to control the

temperature of your sleeping environment

at the beginning middle and throughout

the night and when you wake up in the

morning I've been sleeping on an eight

Sleep mattress cover for nearly 3 years

now and it has dramatically improved my

sleep if you'd like to try eights sleep

you can go to 8sleep.com

huberman to save $150 off their pod 3

cover eight sleep currently ships to the

USA Canada UK select countries in the EU

and Australia again that's 8sleep.com

huberman today's episode is also brought

To Us by Roa Roa makes eyeglasses and

sunglasses that are of the absolute

highest quality I've spent a lifetime

working on the biology of the visual

system and I can tell you that your

visual system has to contain with an

enormous number of challenges in order

for you to be able to see clearly under

different conditions Roa understands

this and designed all of their

eyeglasses and sunglasses with the

biology of the visual system in mind now

Roa eyeglasses and sunglasses were

initially developed for use in sport and

as a consequence you can wear them

without them slipping off your face

while running or cycling and they're

extremely lightweight Roa eyeglasses and

sunglasses are also designed with a new

technology called float fit which I

really like because it makes their

eyeglasses and sunglasses fit perfectly

and they don't move around even when I'm

active so if I'm running and I'm wearing

my glasses they stay on my face most of

the time I don't even remember they're

on my face because they're so

lightweight you can also use them while

cycling or for other activities so if

you'd like to try Roa glasses go to Roka

that's

r.com and enter the code huberman to

save 20% off your first order again

that's

r.com and enter the code huberman at

checkout today's episode is also brought

To Us by betterhelp betterhelp offers

Professional Therapy with a licensed

therapist carried out online now I've

been going to therapy for well over 30

years initially I didn't have a choice

it was a condition of being allowed to

stay in school but pretty soon I

realized that therapy is extremely

valuable in fact I consider doing

regular therapy just as important as

getting regular exercise including

cardiovascular exercise and resistance

training which of course I also do every

week the reason I know therapy is so

valuable is that if you can find a

therapist with whom you can develop a

really good rapport you not only get

terrific support for some of the

challenges in your life but you also can

derive tremendous insights from that

therapy insights that can allow you to

better not just your emotional life and

your relationship life but of of course

also the relationship to yourself and to

your professional life to all sorts of

career goals in fact I therapy is one of

the key components for meshing together

all aspects of one's life and being able

to really direct one's focus and

attention toward what really matters if

you'd like to try betterhelp go to

betterhelp.com huberman to get 10% off

your first month again that's

betterhelp.com

huberman and now for my discussion with

Dr EJ

chelski Dr EJ chelski welcome good to

see you for the audience we are friends

we go way back EJ has been a few years

or more ahead of me in the science game

and the best way to describe you and

your work EJ is you're an astronaut you

go places no one else has been willing

to go before he developed new

technologies in order to do that all

with the Bold mission of trying to

understand how the nervous system which

of course includes the brain works and

how to make it better with engineering

so today we are going to get into all of

that but just to start off and get

everybody on the same page maybe we

could just take a moment and talk about

the brain and nervous system and you

know what it consists of that allows to

do all the sorts of things that we're

going to get into like see things in our

environment and respond to those things

in our environment so at risk of

throwing too much at you right out the

gate what's your one to five minute

version of how the brain

works oh I don't have a one to five

minute version of how the brain works

but I can tell you how I think uh vision

is initiated in the brain and

um you and I go back a long way so we

have a lot of common understanding about

this but I'll narrate it from scratch if

if that makes

sense um so vision is initiated in the

retina uh of the eye which is a sheet

neural tissue at the rear of the eye

that captures the light that is incident

on the eye that comes in through the

through the eye transforms that light

into electrical signals processes those

electrical signals in interesting ways

and changes them up and then sends that

visual information to the brain where it

is used to bring about our sense of

vision and uh you you asked me about the

1 to five minute version how the brain

works I don't know but I do know that

the brain receives all these patterns of

electrical activity coming out of these

nerve cells and the retina and somehow

assembles that into our visual

experience whether that be responding to

things coming at us or our circadian

rhythm

that that govern our sleep and behavior

or identifying objects for prey or

avoiding Predators or appreciating

Beauty and what we know is that the

brain receives a fantastically complex

set of signals from the retina and puts

that all together into our visual

experience and we are very visual

creatures obviously so I think that's a

big part of how the brain works because

so much of what we do revolves around

vision revolves around how the brain

puts together these signals coming out

of the retina and I would love to

understand how that works at the moment

I don't uh and what we're trying to do

is get a really complete understanding

of how that begins in the retina and

then how we can restore it and those who

have lost

sight why focus on this issue in the

retina this thin set of layers of

neurons that line the back of the eye

what why why explore there I mean

obviously there are centers within the

brain that of course contain neurons

nerve cells that are involved in Vision

if one wants to understand visual

perception and I agree by the way that

visual perception is one of the most

dominant forces in the quality and

experience of our

life why focus on the retina why not

focus on the visual cortex or the visual

thalmus I me what's so special about the

retina well we have to focus on all of

it because understanding the retina

won't give us a full understanding of

how all this works obviously and if you

don't have your visual cortex and visual

Thalamus you won't see but if you don't

have your retina you also won't see you

won't even have a chance to see so I

focus on the retina because um I enjoy

the possibility that we can really

understand a piece of the nervous system

in my lifetime in our lifetimes we can

understand it so well that we can build

it replace it restore its function

that's farther off in this in the

central regions of the brain it's going

to be quite a bit harder um I find

satisfaction in really understanding

something so well that I can write down

in a mathematical formula what it's

doing that I can test my hypothesis up

and down and yes we really get how this

little machine works and that I can

engineer devices to replace the function

of that circuit when it's lost that to

me is just deeply satisfying but there

also has is a really fundamental role

for people who want to go and do more

exploratory work in the visual brain as

you mentioned in the visual cortex and

the phalus and other places because

ultimately those retinal signals won't

lead to anything if those areas aren't

putting it all together to govern our

perception and our ultimately our

Behavior so let's talk about the retina

in its full Beauty and

detail three layers of cells line the

back of the eye like a pie

crust somehow take light that comes into

the eye lens focuses that light if it

doesn't do that well we put lenses in

front of our eyes such as contact lenses

or

spectacles and somehow takes that light

and transforms it as you said into

neural signals and processes that within

the retina so let's take a deep dive

into the retina and do so with the

understanding at least my understanding

is that

in part thanks to your work and the work

of others this is perhaps the best

understood piece of the brain yes I

think it's a solid argument that it's

the best understood piece of the brain

and uh we'll turn back to that in a

minute so um the retina begins with a

sheet of cells called the photo receptor

cells that are highly specialized these

are cells that essentially don't exist

anywhere else in the brain and what they

do is transform light energy into

electrical signals in neurons very

specialized very uh demanding cells they

require a lot of Maintenance and they

die relatively easily which is what

gives rise to some some of the forms of

blindness those are the you might call

them pixel detectors they're tiny cells

called photo receptors that each one

captures light from a particular

location in the

world that sheet of cells has done that

initial transduction process where light

is converted into neural signals that

the brain can then begin to work with

the second layer is responsible

for processing adjusting changing mixing

and matching uh comparing signals and

different neurons many complex

operations that we're still trying to

understand and consists of dozens of

distinct cell types that extract

features if you will of the visual World

from the elementary pixels represented

in the photo receptor cells so that

second layer is receiving the input from

that sheet of photo receptors and

picking stuff out of

it the third layer of cells is the

so-called retinol gangan cells that's

the only uh term that I'd like to

probably will come up repeatedly in this

conversation uh so for your for your

viewers and listeners um these retinol

ganglin cells are the ones who are

responsible for taking the signals that

are there in the retina and sending them

to the brain so that the process of

vision can begin they are the The

Messengers if you will from the retina

to the brain the retinal ganglin cells

and there are about 20 different types

in humans um are again feature

extractors they pick out different bits

and pieces of the visual scene and send

interesting stuff to the brain trying to

leave out the uninteresting stuff and

the 20 or so cell types all pick out

different types of information from the

visual scene you can sort of think of

them as Photoshop filters each cell type

in the retina um again about 20

different gangin cell types each type

represents the full scene the entire

visual world but picks out different

features such as some cells pick out

spatial detail tiny little Points of

Light almost some cells pick out and

Signal information about things that are

moving in the visual World some cells

pick out information that's been

captured about different wavelengths

from the photo receptor cells and there

thereby giving us our sensations of

color and probably more things in those

20 different gangin cell types that we

don't fully

understand the result then is that the

retina has this sort of a representation

of the visual world but it has 20

different representations not one it's

not one picture that comes out of the

retina and gets sent to the brain no no

no it's 20 different pictures and you

can think of maybe as 20 different

photoshopped pictures but one of them

has the edges highlighted one of them

has the colors highlighted one of them

has movement uh encoded in it and these

somehow these filters send the

information to many different targets in

the brain and our brain puts it off Al

together and then we have a cohesive

sense of the visual World which is the

remarkable feature that we really don't

understand

amazing is it fair for those that don't

work with Photoshop to think about these

um different Photoshop filters perhaps

as like different movies of the visual

world one movie contains the outlines of

objects and people and things another

movie is showing the motion of blobs in

the environment meaning whatever is

moving environment is kind of just

represented as blobs another movie is

just the color in the environment

another and then all of those what I'm

calling movies are sent into the brain

and then the brain somehow combines

those in ways that allow us to see each

other and see cars and objects and

recognize faces is that is that one way

to think about that that's exactly how I

think about it maybe it's a better way

to say it no I I I like the Photoshop

filter uh analogy I just for those that

don't work with Photoshop um you know I

I just think that the movie analogy

might might be a decent alternative how

the retina works is an example we think

of how all sensory systems work there's

an initial representation in a

specialized cell type that is it is that

is responsible for and capable of

extracting physical features from the

world and then neural circuits in the

brain use that information in different

ways to grab stuff out of the visual

world in the auditory system there's the

the sound world is represented also in

specialized cells that capture sound

energy and transduce that into neural

signals and then subsequent p uh stages

of processing in the auditory system

pick out different features of our

auditory world like the frequency how

high or how low a tone is right the

direction it's coming from right the

movement of it uh how loud it is

different features are extracted so the

we think the visual system is just an

example of how the external world is

represented in our brain and of course

in some sense a a philosophical approach

to the brain is really saying well

there's the sensory world and then

there's the actions we take and there's

almost nothing else that we really know

other than those two things how the

sensory world comes in and then finally

it results in our action that's what our

brain is about because vision is so

important for people I find it

absolutely compelling and fascinating I

mean as an example as you know well many

people study rodents

uh to understand how different aspects

of the brain work and um you know

rodents are interesting animals and do

all sorts of really cool things but they

interact with the world differently than

we do they in in a lot of ways they

sense by smelling they they identify

objects by smelling and they navigate

with their whiskers in to a large extent

we don't do any of that you don't

navigate with your whiskers at least I

don't think you do you don't recognize

me when I walk in the room by my smell

no you use for all that and we humans

use Vision so it's a really fundamental

aspect of who we are as biological

creatures I wonder if just for sake of

entertainment uh we could think about

how the human retina and therefore

Vision in our species differs a little

bit from some extreme examples of vision

in other species not to make this a

comparative um or Zoological um

exploration but just to really

illustrate the fact that the specific

cell types within our

retinas create a visual representation

of the outside world that can be and

often is very different from that of

other species um for instance or at

least my understanding is that the

mantis shrimp sees I don't know 60 to

100 different variations of each color

that we are essentially blind to because

their photoreceptors can detect very

subtle differences in red for instance

long wavelengths of light what most

people refer to as red um pit vipers

cons sense heat

emissions essentially with their eyes

but also other organs um and on and on

you know it I raise this because I think

the the human neuro retina is such an

incredible example of extracting

features from the visual world that then

we recreate but I think it's also worth

reminding everyone and ourselves that

it's not a complete representation of

what's out there like there's a lot in

light that we don't see because our

neuro retina just can't turn it into

electrical signals right you want to

give some examples of what we can't see

and if any um particular examples from

the Animal Kingdom Delight you feel free

to throw those out well one thing you

mentioned color um we experience a rich

sensation of color when we look at the

world and say wow I see all these colors

that's immediate

and that's just how we talk about it but

in fact we have very little information

about color color is a very

high-dimensional complex thing or

wavelength I should say wavelength

information really is about how much

energy there is in the light around us

at different wavelengths we only have

three sort of snapshots of that in our

retinas with the three different types

of photo receptor cells that are

sensitive to different wavelength

different bands of the wavelength

Spectrum three is not a lot as you just

said other creatures have many more ways

of capturing wavelength information and

one way you can verify for yourself that

we just have three is to realize that if

you look at your TV there are only three

primaries on your TV there's a red

there's a green and there's a blue

that's it and from those three primaries

the entire richness of the experience on

your TV set is uh composed so with just

those three things you basically are

able to create any human visual

sensation well the mantis shrimp would

be like that's nothing there's so much

more stuff out there that's not

represented on this TV if you could

speak to the mantis shrimp another thing

we we maybe don't see another example of

a difference in the animal kingdom is so

um again taking rodents as an example

one of the one of the things rodents

have to do is to not be hunted by birds

that are coming down toward them

and so it it appears that there are

cells in the retina that

are seem to be quite sensitive to what

to uh looming to something dark that's

getting bigger uh like a shadow coming

from a bird coming down at you we don't

know for sure that this is exactly what

causes animals to avoid being eaten by

birds but there's there's interesting

evidence in that direction that's not

really a big thing for us for humans as

far as I know we're not typically hunted

by huge birds so that's not a thing we

need and I think that's that's where

comes back to where you were headed if I

understand right um which is we occupy

different biological niches we and the

Manta shrimp and the rodents and our

visual systems reflect that we have

different stuff that we're looking for

in our visual environment than other

creatures are and so our eyes are

different and that's one of the reasons

that we emphasize work on the human

retina um as opposed to other certain

other animal species that would be less

clearly relevant to the visual

experience you and I have so let's talk

about these incredible experiments that

your laboratory has been doing for

several decades now I've had the uh

privilege of sitting in on some of these

experiments and they are very involved

um to say the least if you could just

walk us through one of these experiments

I think the audience would appreciate

understanding what goes into quote

unquote trying to understand what's

going on in the electrical activity of

these specific retinal cell types the

retinal gangling cells in particular um

you know what does this look like you

know you're in your laboratory at

Stanford and you get a phone call

someone says I got a retina what happens

next we scramble like crazy we drop

everything we're doing cancel all our

appointments and get ourselves ready for

48 hours of non-stop work down in the

lab getting as much data as we possibly

can from the retina the most exciting

example of what you just said is when we

get a human retina when for example

there's a a donor who has died and the

retina is available for research we jump

at that opportunity how soon after uh

the person is deceased do you need to

get the eye Globe the eyeball in order

to get the retina in a condition that

would allow you to record electrical

signals from it a few minutes just

you're waiting in the hospital um typic

the way we typically get these eyes is

from brain dead individuals so people

who are legally and medically dead but

they're hearts are still pumping and

therefore their retinas are still alive

and functioning when when those

individuals uh are used their the bodies

of those individuals are used for organ

donations we can benefit from that organ

donation setup that organ distribution

centers do to save many people's lives

and also to promote research so we

sometimes get those retinas and that

begins the experiment for us um I'm

gonna ask for a few more details here

just to put the picture in people's

minds and not to be gruesome I just

really want people to understand what's

involved here so you'll get a call we've

got a a patient who is soon to be

deceased um they've consented to giving

their eye Globes their eyeballs for

research so that you can study the human

retina y um is it you who goes over and

takes the eyes out does somebody do that

or hand them to you in a bucket of ice

I'm sorry if I'm making people queasy at

all but th this is folks how uh one goes

about trying to understand understand

how the human brain works absolutely and

this is also how you go about donating

your heart so that you can save somebody

else's life who needs a heart transplant

the same incredible organizations that

do the harvesting of the tissue for us

their primary goal is to do that for

organ donations to save lives they save

lives every day these people are

incredible donor Network West is one of

those organizations the one that we work

with um they're really amazing um so

their technicians or a retinal surgeon

will take the eye out give it to myself

or some somebody from my lab who will

bring it back to the lab and we have a

way to keep the eye alive and

functioning just the eye by itself is

this always at Stanford or do you

sometimes travel elsewhere local

hospitals up to an hour away so then you

drive them back we drive them back it's

the retina Express and when when we're

bringing back the retin Express it's uh

again it's all hands on deck in our lab

we are scrambling setting up all of our

equipment getting everything ready

you've been at these experiments they're

intense and they really are really are

48 hour marathons of incredible activity

by really dedicated individuals so um we

we might get those eyes sometimes we get

them at 2 in the morning that's common

and from that to in the morning time um

Begins the experiment so we bring the

eyes back we open them up and we we have

access to the rear of the eye which is

what the where the retina is it's a thin

sheet of neural tissue at the back part

of the eye we hemis the eye cut it in

half so that we can see the back it's

like half of a glow if you will and then

we put in relaxing cuts and lay it out

flat so we can see what we're working

with then we take little segments of

retina out in the subsequent 48 Hours

cut them out maybe a 3X3 millimeter

piece of the retina a little chunk of

retinal tissue and bring it into an

electrophysiology recording and

stimulation apparatus that allow allows

us to interact with it and we do two

types of experiments with that so this

electrophysiological recording and

stimulation apparatus is very custom

built by our physics collaborator who

have developed high-end equipment it

allows us to record and stimulate

through 512 channels simultaneously at

very high density this is pretty

high-end stuff in terms of uh technology

for interrogating and manipulating the

electrical signals in the retina that's

what we specialize in in my lab could I

just ask a question about this device um

I've seen it before uh it's very small

as you mentioned your recording from a

few millimet square of the of the retina

um from this recently deceased patient

um it looks a little bit like a bed of

nails right like tiny little microwires

all arranged very Clos to one another

you got the retina laying down on top of

it and that bed of nails can extract

meaning record the electrical signals

that are coming out of the retinal

gangion cells that's right and the

retina is still alive so you are in a

position to shine light on it and

essentially um make it behave um um in

the same way it would if it were still

lining the back of a healthy alive

person that is the beauty of these

experiments so because we can keep the

retina alive and happy and because the

retinal ganglin cells the cells that are

the ones that message the visual

information to the brain are on the

surface we can put them right next to

the electrodes and we can record their

electrical activity in other words we

record the signal that those cells would

have sent to the brain if they were

still in the living person and at the

same time as you said we can focus an

image that we create on a computer

display onto the retina so we're

treating the retina if you will as a

little electronic circuit which it

almost is honestly delivering light to

the photo receptor cells so that they

are electrically excited and then

recording the electrical activity that

the retina is sending out if you will

that allows us to study how the retina

Works

normally what we also do with that same

electrical apparatus is turn around and

pass current through those electrodes in

order to see if we can activate those

ganglin cells directly with no light

just electrodes why do we do that we do

that because it allows us to design

future methods of restoring Vision by

electrical stimulation of the retina

which we'll probably talk about in a few

minutes I'd like to take a brief moment

and thank one of our sponsors and that's

ag1 ag1 is a vitamin mineral probiotic

drink that also contains adaptogens I

started taking ag1 way back in 2012 the

reason I started taking it and the

reason I still take it every day is that

it ensures that I meet all of my quotas

for vitamins and minerals and it ensures

that I get enough Prebiotic and

probiotic to support gut health now gut

health is something that over the last

10 years we realized is not just

important for the health of our gut but

also for our immune system and for the

production of neurotransmitters and

neuromodulators things like dopamine and

serotonin in other words gut health is

critical for proper brain functioning

now of course I strive to consume

healthy Whole Foods for the majority of

my nutritional intake every single day

but there are a number of things in ag1

including specific micronutrients that

are hard to get from Whole Foods or at

least in sufficient quantities so ag1

allows me to get the vitamins and

minerals that I need probiotics

prebiotics the adaptogens and critical

micronutrients so anytime somebody asks

me if they were to take Just One

supplement what that supplement should

be I tell them ag1 because ag1 supports

so many different systems within the

body that are involved in mental health

physical health and performance to try

ag1 go to drink a1.com huberman and

you'll get a year supply of vitamin D3

K2 and five free travel packs of ag1

again that's drink a1.com

huberman let's take this moment to talk

a little bit about cell types so um you

mentioned there are about 20 different

types of these retinal gangling cells

what we may refer to in brief as r gc's

so retinal gangling cells rgc's same

thing

and as you mentioned these cover the

entire retina so that if each cell type

is extracting a different set of

features from the visual World motion

color specific colors

Etc that essentially no location in the

world around us fails to be represented

by these cells put differently these

cells are looking everywhere um each

cell

cell type is looking everywhere um so

that if movement occurs in any region of

our visual world we are in a position to

detect it um but maybe we could talk a

little bit about cell types cell types

is such an important theme in the field

of Neuroscience and indeed in all of

biology but it's actually not something

we have talked about very much on this

podcast before either in Solo episodes

or in guest episodes um I don't have any

specific reason for that we've talked

about brain areas prefrontal cortex

basil ganglia anterior mingal cortex and

on and on we' talked about neural

circuits but we've never really talked

about cell types so the gangan cells as

other you let me down no talking about

cell types well but that's why you're

here that's why I'm here that's why

you're here um tell us about cell types

how do you figure

out if you have a cell type how do you

know if it's a cell type or you know is

it the shape is it how it responds um

how do you know if you have a cell type

what what's this about and I want to

just um put in the back of this question

um or rather in the back of people's

minds that um this issue of cell types

is not just an issue pertinent to the

retina this is an issue that is critical

to understanding how the brain works

absolutely it's critical to understand

Consciousness I know a lot of people

like what is consciousness right we're

not going there just yet but uh what are

cell types how do you determine if you

have a cell type and why is this so

important to understanding how the brain

works yeah I mean as as you said as as

far as we understand every single brain

circuit is full of very distinct cell

types those cell types are distinguished

by their genetic expression their shapes

and sizes which other cells they do

contact and which cells they don't

contact where they send their

information to in other parts of the

brain and what they represent and as far

as we know this is true throughout the

brain and it's true in the retina the

different gangling cell types retinal

ganglin cell types about 20 of them Each

of which is looking at the whole visual

scene extracts different stuff this cell

type one extracts one thing cell type

two extracts something else but they all

represent the entire visual scene but

those cell types we know from lots of

beautiful work work that you're closely

connected to and some of which you've

done um those cell types have different

morphology different shapes and sizes

different patterns of gene expression

different targets in the brain they send

their outputs to different places in the

brain so really to study the retina

without understanding cell types you're

kind of lost right away you have to know

what's going on with the cell types

otherwise you can't make sense of this

retinol signal the way we we identify

them in two ways and they're different

for different purposes the the basic way

we identify the different cell types is

their function because we study their

function we study how they respond to

light images and we can clearly separate

them out and in fact it's it's a simple

thing to say but it's really true our

512 electrod technology which you've

seen in our lab and stuff that developed

with collaborators um about 20 years ago

um was crucial for this because with

that 512 electrode technology we could

see many cells of each type and we could

clearly parse them apart from one

another whereas previous studies working

on one cell at a time had great

difficulty doing that so with our

technology with 512 electrodes we record

hundreds of cells simultaneously we say

oh there's 20 of these there are 50 of

those there's 26 of those and here they

are and we can just set them in

different bins and say okay this is

what's present in this retina just what

the information is they're

extracting there's another purpose again

referring forward to the

neuroengineering aspect we need to

identify the cell types not just based

on what visual information they carry

but based on their electrical features

properties electrical properties of the

cells cells as you know neurons are

electrical cells they fundamentally

receive and transmit electrical

information and the way that they do

that has a distinctive electrical

signature that turns out to be super

important for developing devices to to

restore

Vision could you explain how you

determine what a given cell type does

its electrical properties um let's just

draw a mental image for people uh the

retina's taken out of this deceased

individual put down on this bed of nails

of electrodes those electrodes can

detect electrical signals within the

gangling cells you are able to shine

light onto the retina and see how the

retinal gangling cells respond meaning

what electrical signals they would

transmit to the brain if they were still

connected to a brain they're not

connected to a brain in the experiment

they're sitting there they're trying but

they're

trying I could imagine playing those

cells a movie of I don't know a

checkerboard going where every um Square

on the checkerboard goes from white to

black to gray could do that I could um

play a

cartoon I could um show it uh this

year's Academy Award winner for best

picture like how do you decide what to

show the retina this is a human being's

retina after all uh presumably it looked

at things that are relevant to human

beings until that person died but how do

you determine cell type electrical

signals if you don't know what specific

things to show it I mean you're going to

show it I don't know Disney movies like

what what do you show

it

so what we shown now reflects the fact

that we've built up a lot of information

and our work stands on the shoulders of

many scientists who have studied the

retina for decades to figure out what

different cell types respond to and um

we know that certain cell types respond

primarily to increments of light when

light gets brighter than it was so a

change from a certain brightness to a

higher brightness this particular cell

type fires another cell type fires or

send spikes to the brain when it gets

darker some cell types uh respond

primarily to large Targets in the visual

World other cell types respond better to

small Targets in the visual

World some cell types respond to

different wavelengths of light that we

can identify there exist certain cell

types that are still poorly understood

that respond to movement so we can

tailor visual stimuli to types that we

kind of already know about because of

much preceding

research that's not actually how we do

it in our experiments for the most part

instead we use a very unbiased

flickering checkerboard pattern as it

turns out which is a really efficient

unbiased way to sample many cells

simultaneously so that in a half hour of

electrical recording from a retina we

can figure out what all the 512 or so

cells are that we're recording and know

all of their types and the way we do

that is to play essentially random

garbage TV snow Type image to the retina

for a period of time and determine which

bits of of brightening or darkening or

movement or whatever in that random

garbage activated this particular cell

by looking AC at average across the half

hour recording and saying oh it looks

like this cell was always firing when it

became bright in this region of the

screen that must be an on Cell sensitive

to light in this region of the screen

and so on so we have sophisticated

efficient ways of doing it but it all

comes back to these basic things about

what features in the visual World tend

to cause a given cell to send a signal

to the bra yeah that makes a lot of

sense so you take essentially what you

called random garbage um snow white

black and gray pixels on a

screen yep the retina views that and

then the cells in the retina will

respond every once in a while with an

electrical potential they'll fire as we

say spike it's sometimes called and then

you take sort of a forensic

approach a bit later you look back in

time and you say you know what was the

arrangement of pixels in this random

garbage right before this

cell fired an electrical potential

that's right a spike and then from that

you can reconstruct the preferred

stimulus yeah right the you can say oh

this cell and cells around it seem to

like motion of things going in a

particular direction for instance and um

how do you know know that the cell

doesn't also like a bunch of other stuff

that you

didn't pick up on using

this random garbage yeah two things for

let let me just say for the record we

don't record from these cells that

signal Motion in particular directions

they are an elusive cell type that is

best understood in rodents and other

creatures and not well understood in the

primate as you know although some people

are discovering potential cells of that

type now and have have recently

discovered them okay so let's say cells

that respond

to like spots that are red um you know

that go from dim red to bright red right

yeah so we can go through that colored

TV snow and pick out the cells that

responded to a transition of the kind

you described from darker to lighter or

from Greener to redder or something like

that cells tend to respond to

Transitions in the visual scene rather

than static

imagery um and so we can pick that stuff

out but you asked the question will G is

the TV snow G to capture everything

about what these cells are doing that's

a really important question that I want

to just mention more um quite likely not

that's a scientific instrument it's an

unbiased way to sample a whole bunch of

cells you know first cut look at you

know generally speaking what are they up

to but that doesn't mean we've really

captured their role in natural visual

perception because actually you don't go

through the World perceiving visual snow

you go through the world perceiving

objects and meals and mates and targets

and all these things right so the study

of how the retina responds to more

naturalistic visual stimuli uh in my lab

and in many La other labs around the

world is really get getting off the

ground now and I would say we have

limited understanding um I would say we

know that our simple laboratory

experiments with the TV snow don't

capture the whole story there's more

there're about 20 different cell types

in the retina we have basic

characterizations of seven of them if

you count a certain way um we know that

there are another 15 or so lurking right

behind the curtain that we've started to

sample we don't know what naturalistic

targets they respond to in the visual

life of the animal that's work that's

underway exciting interesting work

because this we know that the r they got

to be there for something one one way to

think of it I'm I'm pretty sure you

think think of it this way too is that

the retina is a highly evolved organ

with a lot of evolutionary pressure for

it to be efficient to have a small optic

nerve sending to the brain it's probably

the case that there's no accidental

stuff sitting around in the retina

that's visal and sending information to

the brain it's probably the case that

this those signals are are all doing

something important for our visual

behavior for our well-being for our

sleep all sorts of stuff and I think and

the field is still trying to figure that

out these these are the big Mysteries I

think that in terms of the retina what

are those signals exactly in all those

different cell types what different

behaviors and aspects of our life do

they

control what is the wildest cell type

you've ever

encountered like what did it do like

what did it respond to that's that's

what I mean when I say wildest um you

know it cells reog gangling cells that

respond to you know increasingly red

portions of the visual scene or

decreasingly green portions of the

visual scene like okay cool that seems

cool like get some you know around the

time of Christmas that that's useful and

it's uh useful on other days of the year

as well but you know given that the

retina is indeed the best understood

piece of the brain and given that you

have 20 cell types 20 isn't 20 million

it's um you know it seems tractable

probably gets understanding it in its

entirety or understanding them in their

entirety excuse me um one would like to

know like what what stuff is are we

paying attention to at the level of the

retina I mean are there like Spiral

sells it like Spiral stuff in the

environment are there sells it like

emojis like what what's going on in

there you spent a lot of time doing this

we do we SP a lot after all you give up

two nights sleep which is kind of

incredible by the way I'll just do a

little uh take a moment here and just

say you know for a guy that's been doing

this for this long with these sleepless

nights um you look pretty good you look

pretty resty

you I tend to go home I go home before

The Graduate students do oh they stay up

they stay up I was used to stay up I

used to stay up until my mid 40s I was I

was in there doing the all nighter type

things got it and um maybe you can help

me figure out my sleep patterns better

yeah yeah we can talk about that this

episode we talk about how to pull all

nighters and still survive done plenty

of those um but yeah like what's yeah

lots at stake here there's a human

retina you know meaning a human gave up

their eyeballs to for this experiment

after they died of

you got many people this these are these

sort of experiments um are very

expensive um a lot of fancy equipment a

lot of salaries to try and figure this

stuff out this is the chief mission of

the national eye institute there's a lot

of tax dollars like this is um in my

opinion as important as the Space

Program probably more important in my

opinion you know restoring Vision to the

blind obviously so what are you finding

in there yeah and and we have the

privilege of being on the front lines of

that funded by the National Institute

and other in tions to be out there

figuring out what's going on in this

human retina I'm with you on that so I

I'll tell you how we go B it these days

there are about seven cell types that we

understand pretty well what they're

doing they're it's not they're not

complicated they just have different

properties color size this kind of thing

temporal properties their timing of

their signals and those seven cell types

we understand pretty well but we're

trying to really nail down the details

why because of neuroengineering for

vision

restoration then there's another I'm

going to say 15 or so and we and the

anatomists the people who study the

shapes and sizes of the cells have long

known that there were more cell types

lurking in the retinal circuitry but

their function has not been known and

because we didn't have many recordings

from them we didn't have electrical

recordings in response to light that

would tell us what they naturally do

we've actually had a breakthrough in the

last few years led by a senior research

in my in my lab named Alexander cing who

has figured out that there's another 15

or so cell types lurking in those

recordings that if we look more

carefully they're there and they have

crazy properties and so the crazy

properties I can tell you about have to

do with the spatial region of the visual

world that they respond to the

well-known cell types that you know and

I know from the textbooks kind of

respond to a circular spot in the visual

world if there's light in this little

circular area they'll fire a spike if

there's no light there they don't care

well okay some cells is not quite

circular some cells respond to the light

that's there and and the difference from

the light that's around it so if it's

brighter than the light that's nearby

then you then you get a big response the

new cell types are more puzzling than

that some of them respond to three or

four blobs in the visual world that's

kind of strange unexpected definitely

unexpected based on the textbooks and

the newest ones are weirder yet some of

them they're their visual response

profiles that is the region of the

visual world that they are sensitive to

light

in almost has a spidery shape almost

like the dendrites of a cell like the

processes of a cell and some of them

have Blobby Light sensitivity they're

sensitive to to light increments here

and decrements there and increments here

and decrements there and some blue light

over here and blue light over here we

don't understand these cells to be clear

the seven that we understand reasonably

well well and not trying to just pin

down and really nail for vision

restoration the sort of first cut at at

cell type specific Vision

restoration those ones don't have these

weird properties they're a little

simpler to understand so we're we're

just but we're just working out all the

details of the timing of the of the

responses and all that these new ones we

don't know what's going on with them so

we're doing experiments those seven cell

types constitute maybe

70% of all the neurons that send visual

information from the eye to the brain so

we think it's a really solid Target for

Vision restoration to work with the

simple

ones and so so when I when I say that I

think that the retina is the best

understood circuit in the nervous system

I'm talking about those seven cell types

which we know a lot about what they do

we really do know a lot it's not done

but we know a lot I'm not talking about

the other 15 cell types which are a

minority of the population but seem to

be doing very strange and surprising

things that are yet to be determined so

there it's a mix we know some really

good stuff and then there's really some

deep Mysteries out there about these

other cell types so we've been talking a

lot about how to understand the signals

that the retina is sending the brain and

I know your lab has done incredible work

in this Arena and figured out a number

of the different signals as you

described some of the features that the

different cell types are extracting just

a moment ago these Blobs of different

colors

Etc what good is this to you know the

everyday person right um What What In

addition to wanting to understand how we

see you know what sort of sorts of

medical applications can this provide in

terms of potentially restoring Vision to

the blind um but perhaps even larger

theme is this notion of neuroengineering

right taking this information and

creating devices that can help us help

our nervous system function better maybe

even function at superphysiological

levels I know there's a lot of interest

in this these days um in part due to

neural link right because elon's out

there front-facing very vocal about his

vision of the no pun intended of you

know chips being implanted in people's

brains that would allow them to be in

conversation with you know 100 people at

the same time just by hearing those

voices in the head maybe filtering

things out so it doesn't sound like a

clamor of a hundred different voices um

perhaps giving people super memory I

mean you know sky the limit no no one

really knows where this is all headed

you're working in what we call a very

strain system where it has specific

properties that you're trying to

understand and once you understand those

you can start to think about real

applications of like what's possible

like could you create a visual system

that um Can extract more color features

from the world that no other human can

see um can you restore um pattern Vision

to somebody who is essentially blind

independent on a cane or a dog or you

know God forbid can't even leave their

house because they can't see anything at

all you know where is this headed what

is the information useful for and

perhaps we should frame that first

within the medical rehabilitative

context of repairing uh or restoring

Vision rather and then get into the more

kind of um uh sci-fi type

neuroengineering stuff absolutely yeah

this this really is my passion these

days turning that corner continuing to

figure out the mysteries in the retina

but also saying wait a second we

actually know quite a lot about this

shouldn't this be the first place that

we can solve problems like restoring

Vision restoring function or augmenting

our function I think it should be the

concept of how to do this is

straightforward and not invented by us

in any way and that is the following one

of the major sources of blindness in uh

the Western world is uh loss of the

photo receptor cells that capture light

macular degeneration and retinitis

Pigmentosa are two well-known ailments

that give rise to to vision loss and the

vision loss is because the cells that

capture light in the first place that we

talked about earlier die off so you're

no longer sensitive to light and then

you're blind the concept is that you may

be able to bypass those early sections

of the retina that capture the light and

process the signals and

instead build an electronic implant that

connects up directly to the retinal

ganglin cells and this electronic

implant would would do the following it

would capture the light using a camera

which is relatively easy it would

process the visual information in a

manner similar to the way that the

retina normally does as similar as

possible and then it would electrically

activate the retinal gangin cells by

passing current and causing the gangin

cells to fire spikes and send those

spikes to the brain and if we do this

really well we can essentially replace

those first two layers of the retina and

piggy back onto the third layer and say

okay we're just jumping right into that

third layer we're going to force those

ganglin cells to send reasonable visual

signals to the brain and then the brain

is going to think it got a natural

visual signal and proceed accordingly

that's the concept this is not our idea

people have had this concept for decades

and some people have even started to

make it work in in human patients and

what I mean by that is implanting

electrodes on the retina stimulating and

causing people who've been profoundly

blind for decades to see visual

sensation blobs and streaks of light in

their visual world that that are

reproducible so so that's happening now

that's happened people who were at once

blind yep are able to see objects are

able to see crude blobs and flashes of

light in ways that allow them to

navigate their world better a little bit

a little bit avoid a coffee table maybe

or at least see a bright window in a

dark wall and be able to point to that

bright window or the bright doorway in a

dark wall something like that so it's a

glass half full half empty story that I

want to turn that I'd like to turn

attention to in this conversation cuz I

I think it's very exciting um yes you

can see stuff by artificially

electrically stimulating the ganglin

cells and you can see stuff that

actually helps you interact with your

world a tiny little bit so great that's

the glass half full the glass half empty

is it's nothing remotely resembling what

we understand as naturalistic Vision

where we see spine spatial detail and

color and objects and can navigate

complex environments and all that stuff

no chance you can't do anything REM

remotely like that you can see that

there's a bright doorway over that way

and turn toward it which is a helpful

step in your Human Experience but

there's a long way to go so the question

is why does this existing technology

fail to give us high quality Vision what

what's needed to give us high quality

vision and this is the piece I'm really

passionate

about it turns out that the the devices

that have been implanted in humans so

far by pioneering bioengineers who did

really hard stuff were were fairly

simple devices that treated the retina

as if it's a camera that is just a grid

of pixels and they put a grid of

electrodes down there and they

stimulated it according to the pixels in

the visual world and thought well

hopefully that will cause the retina to

do the right thing and send a nice

visual piece of visual information to

the brain and initiate Vision

Unfortunately they left the science on

the table and this is actually what I'm

dedicating the next phase of my career

to bringing the science that we know

that we talked about to the table for

vision engineering and in particular the

fact that there are there really are 20

or so distinct cell types and they send

different types of visual information to

the different targets in the brain I

like to think of them a little bit as uh

an orchestra playing a symphony each

different cell type has its particular

score the violins do one thing the obos

do something else it's a very organized

signal coming out of the retina

presenting to the brain this complex

pattern of electrical activity that the

brain assembles into our visual

experience well current retinal implants

unfortunately are too crude to do

anything like that the the conductor has

just scattered the sheet music

everywhere and people are playing

whatever it's cacophony okay you can

maybe recognize a tune in there a little

bit sometimes navigate toward a doorway

but you're not going to get the full

richness of the experience by ignoring

the different cell types and I'm so

passionate about this in part for

reasons that a little bit are similar to

your reasons for doing this kind of work

that you do which I think is

great which is I I feel we have a

mission to give back as scientists to

take all this stuff we've been talking

about because we find it so interesting

and

cool and to deliver something to the

society that has allowed us to explore

these fascinating areas and in our case

it's in my the case of my lab and what

we've done it's to turn around and say

wait a second we understand that there's

these different cell types we understand

a lot about what they do none of this

information appears in current epir

retinal implants can't we do better by

using the science to restore Vision in a

way that respects the circuitry of the

retina that's what we're trying to do

and the mismatch is intense um I I told

you when we were adding

before that nothing that we have learned

about the retina since the founding of

the national Eye Institute in

1968 is incorporated into the existing

retinal implants nothing we've learned a

ton about the retina your research was

funded by the national Eye Institute my

research is funded by national ey

Institute a fantastic organization that

allows us to learn all these things how

is this showing up in the

neuroengineering to restore Vision to

people well currently it's not

and so we're trying to do that now doing

that turns out to be hard and maybe

we'll talk about that it's a it's a

technological feat that's really

challenging you have to build a device

that you can implant in a human that can

recognize the distinct cell types see

where they are stimulate them separately

from one another and conduct this

Orchestra to create a musical score that

reasonably closely resembles the natural

one that's what we're all about doing

and as it turns out and maybe we'll talk

about that separately that mission of

being able to restore the patterns of

activity that the retina normally

creates also has extremely exciting

spin-offs in three directions one of

them is understanding better how the

brain puts the signals together that's

research for the brain the second one is

augmenting Vision creating novel types

of visual Sensations that weren't

possible before and maybe doing

something along the Elon Musk lines of

delivering more visual information than

was ever

possible and third figuring out how to

interface to the brain more broadly

because as you and I know the structure

of the retina is very much

representative of the structure of many

brain areas and if we're going to figure

out how to interface to the visual

cortex we darn well bitter figure out

how to interface to the retina first

that's what we're all about doing in my

lab these days is figuring out how to do

that well that's a mixed science and

engineering effort we've done about 15

years of basic science on that how do we

stimulate cells how do we recognize

cells how are we going to build a device

that does all this and talks to the

cells in this way and I can go into lots

of gory detail about it um but that's

what we've been doing the basic research

on and the last few years we've worked

at Stanford with um fantastic Engineers

from various disciplines electrical

engineers material scientists others to

figure out how to put together the

pieces and build an implant that can do

this in a living human so is the idea to

build a robotic retina to build a a

essentially an artificial retina that

could be put into the eye of a blind

person or even put into the eye of a

sighted person that would fundamentally

change their ability to see or in the

latter case uh enhance their ability to

extract things from the visual world

that they would otherwise not be able to

see like like seeing twice as far um

getting you know hawk-like resolution of

the visual World um yep or which that

would be cool yep could be distracting

yep like I'm not sure I want to see the

the fine movements of a piece of paper

in somebody's notebook from across a

cafe as they flutter the pages um but

you might want to for a moment there

might be a moment when you want to and

if you have an electronic device that

you can control that you can dial in to

sense different aspects of the visual

World Under You Know by by your choice

you might be like yeah that's pretty

cool I want to be able to do that right

now there's an example I like to give

which I think maybe is is helpful for

interpreting what we're talking about

when we say being able to do more things

with the optic nerve you gave the

example of many voices and stuff here's

an example that I like we know that we

can drive down the road and have a phone

call

handsfree and do that quite safely

pretty safely right and you're why

because you're tapping in you've got two

types of signals coming into your brain

your visual signal of the cars on the

freeway any one of which could kill you

in an instant so it's

important and the sound coming into your

ears which carries the voice of your

girlfriend that's telling who's telling

you something that you're interested in

hearing and these are different parts of

the brain processing this information

and so you can do both of these things

at the same time because there's no

interference one part of the brain is

working doing one thing another part's

doing something else you're good what

you can't do is to read your texts and

drive down the freeway at the same time

that's not good because now that visual

system of yours that needs to be

detecting these fast moving dangerous

objects is being distracted by looking

at the text and you might die and some

other people might die with you so I see

a lot of people texting and

driving yeah that's why I like to point

out this example to remind people you

can't do it well it's like it's like you

can't do it well you probably talk yeah

it used to be you know we'll just take a

brief uh tangent here into this topic um

a few years back there were there were a

lot of news articles a lot of discussion

about texting and driving a lot of

attention to getting people to stop

texting and

driving I've seen a few people pulled

over for texting and driving before but

I would say texting and driving is

rampant reading what's on one's phone y

while driving is rampant yep all you

have to do is be on the freeway here in

Los Angeles look in the car next to you

yep look where they're looking and

people like reading and texting while

driving presumably when they're doing

that they're just using their peripheral

vision to detect any kind of motion and

um no doubt this has caused the deaths

of many people yeah change lanes get

away from them and you know just just

like that other driver so so here's the

thing um and this is this is uh I say

this a bit tongue and cheek but it's

it's sort of a real example it may be

that if we harness the different cell

types in the

retina to deliver different visual

information to different cell

types like the image of the text on your

screen to a certain cell type that you

know the so-called

cells or the motion of the objects in

the visual scene the cars to a different

cell type cells by the way folks

because they're very very small yes and

by named by anatomist decades ago so we

carry that nomenclature forward sure um

and the parasol cells which are

different cells that you can potentially

encode the movement of the cars in the

parasol cells and if those two systems

are operating independently which we

sort of think they may be from research

that you know very well from your

extensive studies and vision

um maybe we can do those two things

safely at the same time now by the way

that's not my research goal to text and

drive at the same time just to be clear

but it's a very tangible real world

example of if we do really have parallel

Pathways that can be modulated and

controlled independently of one another

this opens the door to streaming all

kinds of visual information in parallel

into our very high band with visual

systems that wasn't possible during

Evolution because we didn't have control

over the cell types so I think of that

as the world of visual augmentation and

it starts to get interesting if the

different cell types are behaving in an

independent way and when they transmit

visual information to the brain now how

are we going to figure that out well we

need a device that can stimulate the

different cells independently and then

study that to see whether people can and

can do this kind of thing right what's

that device the artificial retina the

same implant I'm telling you about that

can restore Vision to people because

it's an electronic device that can up in

the activity in the different cell types

that same device is what we can use as a

research instrument to understand if the

different pathways are parallel if the

signals interact with one another and

explore how the brain receives that

information and then we can use that to

explore can we augment vision and allow

allow ourselves to have new visual

Sensations that we don't even know what

that would look like we we we don't even

understand what it would look like to us

to see those Sensations but it might be

able to deliver lots of information to

our brain and if we can do all those

those things then we can take that same

set of tools and Engineering

Technologies into the brain to access

different cell types as well I'd like to

just take a quick break and acknowledge

one of our sponsors inside tracker

inside tracker is a personalized

nutrition platform that analyzes data

from your blood and DNA to help you

better understand your body and help you

reach your health goals now I've long

been a believer in getting regular blood

work done for the simple reason that

many of the factors that impact our

immediate and long-term health can only

be addressed that is can only be

measured with a quality blood test now

one issue with many blood tests out

there is that you get information back

about lipid levels hormone levels

metabolic factors Etc but you don't know

what to do with that information with

inside tracker they make it very easy to

understand your levels and what they

mean and specific actionable items that

you can undertake in order to bring

those levels into the ranges that are

optimal for you insid tracker also

offers insid Tracker Pro which enables

coaches and health professionals to

provide premium and personalized

Services by leveraging inside tracker

analysis and recommendations with their

clients if you'd like to try insid

tracker you can go to insidetracker

docomo and you'll get 20% off any of

insid tracker's plans again that's

insidetracker

docomo here of of what you've done um

you started off with the understanding

that the neural retina is perhaps the

best place to try and understand how the

brain works because of its Arrangement

the cell types Etc you spend a number of

decades doing these wild experiments um

on human retinas and other retinas um

recording the different cell types with

these high density What I Call Bed of

nails two and a half decades I'm not

that two and a half decades um it's your

uh robustness that matters EJ um and you

have plenty of it um you figure out what

the cell types are so then you gained an

understanding of how light is

transformed into different types of

electrical signals that encode different

features in the visual scene then comes

the challenge of developing

neuroengineering tools to try and

stimulate the specific cell types in a

way that more or less mimics their

normal patterns of activation like not

um activating a huge piece of retina so

that you know the cells that like

increases in redness are also being

stimulated with the cells that like you

know edges in a way that would create

some shoy like crazy representation of

the outside world no you want the same

Precision that light stimulation of

these cells in the intact human eye

provides in this explanted retina this

retina on this bed of nails but then a

device that essentially can mimic what

the retina does and you needed to do all

that earlier work understanding like

what does the normal retina do what does

the healthy retina do in order to try

and develop this prosthetic device to

either restore Vision

or because it puts puts you in the

position of being able to stimulate

cells however you want in theory you

could create a situation in a human

where the cells that respond to um I

don't know outlines of objects are are

hyperactive so that you

know the person could um effectively see

objects in one's environment better than

anyone else yeah could perceive I know

motion is a tricky one but motion better

than anyone else could see detail in the

visual world that no one else could

detect we're not talking about turning

people into mantis shrimp um but the

analogy works because mantis shrimp can

see things that we can't and vice versa

and so what you're talking about here is

neural augmentation through the use of

engineering yep and we often do talk

about it as sci-fi because the Sci-Fi

writers have been talking have been you

know writing about this for decades it's

not sci-fi anymore it's sigh it's

straight up sigh right now it's really

we just need to build the

instrumentation and start working with

those experiments to figure out how to

make make it work I think we have a

responsibility to do this because this

is the way we take this kind of

information all that's been learned

about the visual system by the national

Eye Institute since since 1968 and all

the people that it's funded to do this

research and turn the corner and make a

difference for Humanity with it and I I

assume and think that Humanity will be

leveraging nervous system knowledge to

build all sorts of devices that we can

interface to the world with I think you

know I don't know Elon Musk but I think

he's right about that that that's where

we're

headed it should be done well it's

important to do it well um we will

hopefully be more connected to truth in

the world if we're able to build devices

that give us better Sensations more

acute understanding of what's going on

out there better abilities to make

decisions and all that let alone just

see stuff so that Frontier of developing

Technologies to allow our brains and our

visual systems initially and then other

parts of our brains to do things better

is unbelievably exciting it is SciFi but

I just want to emphasize I think it's

the responsible way to go to think about

how to do that well all technologies

that we develop can be used for good or

for ill and I'm sure some of your

listeners who are a lot a lot of very

passionate thinking people out there

thinking about neuroscience and the

implications worry what does this mean

we're going to be introducing electronic

circuits in our brains to do stuff yeah

well we will it's pretty much clear that

Humanity will do that and

so in any technology development you

have to think about well how do you do

it well how do you do it for good there

there are popular movies right now about

technology development such as

understanding the structure of the

atom and that technology development can

be used for good or for ill to blow up

cities or to save

civilizations how's it going to go well

I think I think as scientists we're

responsible for advancing that in a

thoughtful and meaningful way I think we

can do this in the retina it is the

place to start it's the and you know you

I'm curious what you think actually as a

scientist your background is in this

field or very close field to mine I know

you speak with all sorts of scientists

on this podcast um but this is pretty

much your field or very close not the

neuroengineering part but understanding

the retina I'm curious if you agree that

this is the place to start doing this

stuff the first guest ever asked me a

question on this podcast during a guest

interview um I think this would be a fun

place to both answer and Riff on this a

little bit because um first of all I

think the retina is absolutely the place

to start because we understand so much

of what it does what the cell types are

but maybe by comparison a different

brain region the hippocampus which is

involved in the formation of memories

and other stuff but formation of

memories about what one did the previous

day what one did many years ago Etc is

an area that I think anytime the

conversation about neural Prosthetics or

neuroengineering or neuro augmentation

comes up people think oh wouldn't to be

cool to have like a little stimulating

device in the hippocampus and then if I

want to remember a bunch of information

from a page or from a a lecture I just

stimulate and then voila all the

information is batched in there um while

that's an attractive idea I think it's

worth pointing out right now that sure

there is a pretty decent understanding

of the different cell types in terms of

their shapes some of their electrical

properties of the hippocampus but there

is in no way shape or form the depth of

understanding about the hippocampus and

what the individual cell types do and

what the different layers of it do that

one has for the neural retina so what

we're really saying is if you stimulate

the hippocampus you'll likely get an

effect but it's unclear what the effect

is and it's not clear how to stimulate

that's to me the best reason to focus on

the retina because you know what the

cell types are thanks to the work of

your laboratory and many other

Laboratories you know what sorts of

stimulation matter and it provides the

perfect test bed for this whole business

of neuro augmentation neuro neuroeng

engineering I think there's also a

bigger discussion to just frame this in

which is so much of what we discuss on

this podcast with guests and in Solo

episodes relates to things like dopamine

neur neuromodulator serotonin and

everyone is interested in these things

because they can profoundly change the

way that we perceive and interact with

the world but one only has to look to

the various Pharmaceuticals that

increase or decrease these

neuromodulators and know that indeed

those Pharmaceuticals can be immensely

beneficial to certain individuals I want

to be clear about that but that whatever

quote unquote side effects one sees or

lack of effect over time is because

those receptors are like everywhere over

the around the brain so you can't just

increase dopamine in the brain and

expect to only get one specific desired

effect yes so the reason you're here

today um is not because we both worked

on the retina and it's not because we

happen to also be friends it's because

uh to my mind your laboratory represents

the um Apex of precision in terms of

trying to figure out what a given piece

of the brain in your case the neuro

retina does parse all its different

components and then use that knowledge

to create a real world technology that

can actually tickle and probe and um

stimulate that piece of the brain in a

way that's meaningful right not just

like sending electrical activity in and

that to me is so important I think if we

were going to think about levels of

specificity for manipulating the human

brain to get it an effect you would say

okay crudest would be drugs take a drug

increases serotonin which might bind to

particular receptors let's take the drug

psilocybin a lot of excitement about

psilocybin we know that can lead to

increased connectivity between different

brain Reg at rest there's probably there

is some demonstrated clinical benefit

there's also some potential hazards but

it's very broad scale we don't know

what's happening when the person is

thinking about a uh you know a piece of

moss expanding into an image and a

memory of their childhood or it's like a

million different things are happening

there and then at the other far extreme

is the kind of experiment that you're

talking about stimulating one known type

of retinal

cell seeing what that means for visual

processing or modeling what that means

means for visual processing and then

building a device that can do exactly

that and then maybe ramp it up 20% 50%

and because I think that represents the

first step into okay how would you

stimulate the hippocampus to create a

super memory would you stimulate a

particular cell type in a particular way

and to my knowledge There's You know

despite the immense excitement

about the hippocampus and understandably

so there just isn't that precision and

of understanding yet so uh forgive the

long answer but you know you ask me a

question on on this podcast give a long

answer yeah and it specificity is what

you're talking about and we need

technology to do that to to modulate

neural circuits in a highly specific way

we got to start with the piece of the

neural circuit we understand best and we

have best access to that's the retina

it's sitting right there on the surface

we can get right into it and in install

devices we know so much about it that's

the place to start the place that we

understand build Electronics that is

that is adaptive that senses what

circuit it's embedded in and responds

appropriately it's not just Electronics

you stick in there and it does something

and that's it no it it first figures out

who it's talking to and then learns to

speak the language of the nearb neural

circuitry so a smart device smart device

let's push on that a little bit so

um put a little chip onto the retina

that can stimulate specific cell types

is there a way that it can use AI

machine learning that it can learn

something about the tissue it happens to

be in contact absolutely in the simplest

possible way the device Works in three

simple Steps step number one record

electrical activity which is what we do

in our lab in a room full of equipment

but this is a 2mm size chip implanted in

your eye record the electrical activity

just recognize what cells are there when

they're firing what electrical

properties they have to identify the

cells and cell types in this specific

circuit in this human that's step one

step two stimulate and record so you

figure out oh this electrode activates

cell number 14 with 50% probability this

electrode activates cell number 12 if

you pass a micro amp of current with

this probability and so on you make a

big table that tells you how these

electrodes talk to these cells in your

circuit that's step two calibrate the

stimulation by stimulation and recording

then finally when you have a visual

image and you want to represent it in

the pattern of activity of these cells

you say okay I know from Decades of

basic science what these cells ought to

be doing with this image that's coming

in I know exactly what they ought to be

doing that's what the science has been

telling us we've been studying the

neural code for decades to understand

this I know what they should do use my

device with my calibration where I know

where the cells are I know how the

electrodes talk to them and bing bing

bing bing bing activate them in the

correct sequence that's what I think of

as a smart device a device that records

stimulates and records and then finally

stimulates yes AI is part of that of

course it is because this is a very

complex uh transformation if you will

from the external visual world to the

patterns of activity of these cells it's

not easy to write down in just a few

lines of code or in a few equations it's

complicated and AI is really helpful for

that and learning by stimulating and

recording and aggregating information

quickly so that you can then use the

device that's absolutely a part of the

engineering it let me be clear the AI

doesn't help us to understand it's just

an engineering tool that helps us to

capture what this thing normally does

and then go ahead and execute and make

it do the thing it should normally

do I hope people um will appreciate this

example uh perhaps not you know not but

goodness I don't know 40 50 years ago

but still today uh one treatment for

depression is electric shock therapy uh

a very you know on the face of it

barbaric um treatment but effective in

certain conditions it's still used for a

reason um but it can appear barbaric

right you know people are have like a

bite you know a bite device you know so

they don't um bite through their tongue

or their lips they're you know they're

strapped down and they stimulate the

whole just like stimulate all neurons in

the brain essentially there's a huge

dump of neurotransmitters and

neuromodulators it's like drugs it's

completely non-specific stimulation

effectively probably even less specific

than drugs maybe and yet the clinical

outcomes from electric shock therapy in

some cases are pretty impressive like

people um the brain is quote unquote

reset they still remember who they are

um but presumably through the the the

release of neuromodulators like dopamine

serotonin acetylcholine in a very

non-specific way there has been some

symptom relief in some cases what you're

talking about is really the opposite

extreme um you know before I said

Pharmaceuticals that tickle a particular

neurom modulator pathway would be the

opposite extreme I think electric shock

therapy is probably the the the most um

extreme where is this whole business of

neural prosthesis going outside of the

visual system like right now um I can

imagine that there are little

stimulators in the spinal cord for sake

of restoring movement to paralyze people

um I realize this is not your field but

is are you seeing impressive stuff there

um or is it still really really early

days there's absolutely impressive stuff

uh particularly uh for example people

reading out signal

from the motor cortex or the language

cortex in order to help people to

communicate or to move cursors on

screens in order to interact with

devices these are paralyzed people par

yeah excuse me paralyzed people who

can't interact with technology the way

that we do um and but with their

thoughts can send signals through an

electronic device that can be used to

control a mouse on a screen and have

them connect to the internet that's a

huge deal to be able to have people do

that imagine how life changing would be

to be able to communicate if you

couldn't before so there are wonderful

examples of that you know of them so do

I the work of Krishna shanoy and Jamie

Henderson at Stanford is one beautiful

example over the recent years Eddie

Chang doing stuff now uh you know

neuralink doing doing this kind of stuff

the built on the work of Shino and

Henderson and stuff so that's great um

you know stimulating in spinal circuits

as you said for creating rhythmic

movements that's that's happening

so this is an enormous space and in each

case what you said I think is really

highly relevant that electroshock

therapy you can think of that as look

let's say your computer is not behaving

right you can reboot it might work then

it'll start not working again then you

have to reboot it again well how often

do you want to reboot your computer it

gets a little inconvenient to be

rebooting your computer every five

minutes maybe you want to go in and

actually diagnose this thing and put in

a piece of software that fixes the thing

that was going wrong instead of

rebooting your computer every 5 minutes

right and I think of electric shock

therapy a little bit as a reboot um it's

that level so we want to intervene more

specifically how do you do that well you

have to understand the software in order

to do that you have to have specificity

controlling this thing in your computer

not this this this this or this this

particular thing that's going wrong you

got to interfere with that and change it

and modulate it well that's what

understanding the neural circuit is

about that's what building specific

Hardware that can activate specific

cells is

about that's in in the case of the

retina again it's just so obvious that

it's right in front of us to do this

stuff and it's right in front of us to

take us into augmentation to giving us

better senses a fun example I like this

is it's an interesting topic because the

National Institutes of Health that um

funds a lot of research that goes in

this direction tends to not be

interested in augmenting our senses they

kind of want to they they draw a line

more or less at saying look we're trying

to restore what we were as humans not

create a new kind of

human and that's an interesting question

because I don't think there is a fine

there's an actual line a bright line

between those things I don't think

there's a bright line between those two

things the the finest example I know is

that even in the very crudest visual

restoration devices you have to actively

suppress the infrared sensitivity of

your camera

to not have infrared vision why because

many cameras are sensitive to infrared

light so in other words if you don't put

an infrared filter in front of your

camera you're going to have some

infrared vision maybe it won't help you

very much whatever I'm just trying to

say as soon as you start building

devices to restore Sensations building

electronic devices augmentation is right

around the corner it'll creep up on you

real fast so I I think you you can't

even really draw a

line throughout today's discussion we've

been thinking of the brain as kind of a

the rest of the brain I should say

because the neural retina is two pieces

of the brain extruded out into the eye

Globes during development I like to

remind people of that over and over when

you're looking at somebody's eyes you

are looking at two pieces of their brain

um so no debate about that but most

people don't realize that you'll never

look at anyone the same way again but

this is the reason why you can tell so

much about people's inner state from

their from their eyes somebody who's

sleep deprived it's not just about the

droopiness of their eyelids or the

circles under their eyes there's a

there's an aliveness to the eyes the the

yogis um talk about um people that sort

of show up at the level of their eyes as

opposed to sunk back into their brain

you know these are very um kind of

abstract Concepts but they and there's

some very non-abstract stuff these days

looking at looking through the eye at

the retina the way the op

opthalmologists do and there's a lot of

diagnostic capability just in those

images of the rtina oh right there I'm

glad you brought this up there's um some

uh interesting and increasing evidence

that looking at the neural retina

because it is a piece of the brain with

neurons that have the potential to both

Thrive or degenerate that looking at the

neural retina which one can do with

these new technologies um can provide a

window into um whether or not there are

forms of degeneration such as

Alzheimer's and other forms of neurod

degeneration deeper within the brain

that one can't image directly because of

the the thick opacity of the skull right

so in other words Imaging the eye in

order to determine if someone is

developing in Alzheimer's because you

have a direct view into a little piece

of the brain it's a it's a good signal

it can help you figure stuff out about

what's going on in your brain even

beyond the sunken eyes

absolutely amazing so I think the rest

of the brain piece is is also really

interesting and maybe here we can go

like semi neuro philosophical um you

know that there are clearly parts of the

brain that are involved in um essential

what I call housekeeping functions

regulating respiration you know keeping

us breathing keeping our heartbeating um

digestion um responding to threats in in

some sort of basic way like through the

secretion of adrenaline and and giving

us the potential to move but a lot of

the brain is is um capable of plasticity

and um one wonders if you were to for

instance develop a retinal

prosthesis that would allow me to um see

with twice the level of detail that I

currently can would my adult brain be

capable of dealing with that information

we're talking about twice as much

information coming in same brain tissue

on the receiving end can it make sense

of it do we have any idea if it can make

sense of it are there um are there

experiments that speak to that yeah

that's a fantastic and interesting

question it makes you think about neural

development all over again right and I I

take some inspiration on that from the

work of someone you know Eric nson um

who discovered that there is plasticity

beyond the period periods of time he

discovered many wonderful things but

there's plasticity well beyond the

period of time that we thought that

there was plasticity in certain animals

and in particular that if you make

gradual adjustments to the sensory world

you can exhibit plasticity that you

can't see if you make an Abrupt

adjustment so plasticity is there it

just has to be brought on by more subtle

manipulations that take you from A to B

in little incremental steps and if you

take those incremental steps you can see

the adult

plasticity so coming to your

question is the brain capable of

receiving the kind of extra information

we provided could be that if we just

show up on day one bam try to deliver

twice the visual resolution or whatever

in your visual your visual

system it could be that if we try to

deliver twice the visual resolution on

day one it won't work you'll see It'll

you know it won't look sensible to you

but if we gradually introduce it by the

way that we're dialing up the resolution

we may be able to get there and there

are fascinating studies to be done you

think about Spike timing dependent

plasticity a term that that your viewers

may not all know but is a is related to

how neurons adjust their strength of

connectivity to one another according to

the timing of the signals in those cells

mechanisms like that tell us wow the

brain really cares about the very

precise timing of stuff and to the

degree that that influences the way that

neurons do or don't strengthen their

connections to one another it's so

fundamental in everything from memory to

visual function what have you this

relates to fire together wire together

although um it highlights the Together

part how closely in time do two neurons

need to be active in order for them to

subsequently increase their connectivity

and indeed one of them needs to be

active a little bit before the other one

in order for it to work optimally right

so if it what I what I Envision is that

when we have full control of the neural

code with an electronic implant that can

talk to the cells and do all the things

that I said and we can really control

the neural code coming out of the retina

we can then start to play games and dial

up that neural code very slowly and

teach the remaining brain how to

understand these signals not introduce

some crazy thing from day one no just

gradually teach isn't that how we do

everything well isn't that how

plasticity

works I love the uh subtlety and the uh

rationality of your example because you

know so much of what we see in the

internet and on the news is like you

know chips inserted into the brain to

create super memory or uh you know

conversations between you know 50 people

at the same time without anyone speaking

you know just hearing other people's

Thoughts by way of um you know neural

signals being passed from one to the

next but yet another reason why you're

here today is because you represent the

the the realistic grounded um

incremental approach to really parsing

this whole thing of how the brain works

and how one then goes about um

engineering devices to augment the human

brain and as you just pointed out it's

not going to be done by just sticking

electrodes in and stimulating and seeing

what happens in fact those experiments

were done in the 1960s people like

Robert Heath would put electrodes into

the brain during neur surgery stimulate

and just see what the patient would do

or what they reported thinking you know

nowadays that's done still but with a

lot more precision and intention and

we've moved we've moved far beyond that

by the way I just want to say those were

important first experiments the first

thing got to do Keith was was was a

rather um let's be honest not not the

most Savory character he he embarked on

some experiments that um had a social

agenda to them and was a pretty pretty

uh at least by my read was was not the

kind of person I'd want to spend time

with to say the least but but you're

right those experiments were critical

because like electric shock therapy like

the formulation of drugs that can

massively increase certain neurom

modulators or decrease them they LED to

some level crude but some level of

understanding about how the brain works

which is what we're really talking about

today yep um but you represent the uh as

I said the the astronauts of this

astronauts don't go into space and just

kind of blast off and see where they end

up there's a there's there's math

there's physics uh there's computer

science sensors cameras right looking

where are we about to land here on the

moon is there a crater here or not what

what's around us we we should sense

what's there and then make our decisions

accordingly and our electronic implants

in the brain really we should make them

smart why make them dumb we're smarter

than that we can build implants that can

sense what's around them and change

their patterns of activity

accordingly I I use a metaphor sometimes

if you go to if you go as an American

who doesn't speak Chinese to China and

you start yelling in English maybe

somebody will to to learn which way to

go on the street somebody might

understand you at some point and help

you out but it's going to be a it's

going to be a you know not very

effective way to get around it's way

better if you speak the language you

talk to people and ask ask them where to

go so that's what we need to know we

need to say look we have the science we

have incredible devices we can engineer

we have ai now that even helps us to do

this query of the outside world and turn

it into a smarter instrument make our

instruments smart make them so they know

how to talk to the brain don't expect

that the brain is just going to wrap

itself around your your simple

electronic device no make a smart device

that's what we want to make a smart

rental

implant maybe we could just take a

couple of minutes and talk a little bit

about you and the some of the things

that have led to your choice to go in

this direction

so did you always know you want to be a

neuroscientist from the time you started

college what was your trajectory I

should know this but I you were an

undergraduate at at Princeton at

Princeton that right studying math math

so you could have just done all your

work with a piece of paper and a pen but

you had to try and engineer all these

electrodes okay that's a pen and paper

pen I took a very random route I studied

math as an undergrad I spent a few years

running around playing music and

traveling living a Bohemian life well

tell me more about that oh it was I

basically just told you all I'm going to

tell following the Grateful Dead no not

quite following them but uh it was that

was an important part of the story um

and important part of your personal

development um yes very much so uh free

Expression Dance music creative

exploratory music all that kind of stuff

such a contrast to the the EJ that comes

forward when we're talking about the

Precision of neural stimulation in in

the in specific retinal cell types but I

think it's useful for people to hear

both young and old like um that uh one's

nervous system can be partitioned into

these different abilities like go and

dance and travel and you you weren't

doing anything academic at that at that

time no for a few years I wasn't doing

that programming computers to make a to

make a living and uh then I started I

started three different PHD programs at

Stanford before I simultaneously no no

no in sequence I started in the math PhD

program I learned that was not really

for me and I started in a in an

economics PhD program in the business

school there and I realized after after

less than a year that was not for me um

I worked in a startup company for a

while I did a lot of stuff for a few

years and it took some settling um but

then I decided to go into nurse science

and there were a couple formative things

one is that I had I had gotten a really

formative experience as an undergraduate

from a wonderful guy called Don ready

who taught a introductory Neuroscience

course who was really inspiring Mentor

um and then when I when I was at

Stanford I met Brian wandell my PhD

adviser and I was inspired by him I

thought I didn't know why he was

studying what he was studying but I just

knew I wanted to learn from this man and

I wanted to study with him I just knew

this was this was the person who should

be my mentor based on something about

him yes C can I ask you about these

three PHD programs because I think

people here um you know or or see what

you're doing and and probably imagine a

very linear trajectory but now I'm

hearing you you like tour it around

playing music then you start a PhD

program nope then another one nope then

another one without getting into all the

details I mean were there nights spent

lying awake thinking like what am I

going to do with my life or did you have

the sense that you knew you wanted to do

something important you just hadn't

found the right fit for you like how how

much anxiety on a scale of 1 to 10 10

being total panic um did you experience

at at the Apex of your anxiety in that

kind of wandering am I allowed to go

above 10 like turning up the amp to 11

sure I just think it's really important

for people to hear whether or not they

want to be scientists or not this idea

that people that are doing important

things in the world uh in my view um um

rarely if ever understood that that's

the thing that they were going to be

doing there was some wandering about

that sure seems like it doesn't it yeah

I I I experien the same when I talk to

other people and it seems like that and

for sure for me uh it just took a while

of trying different things to see number

one what I was really good at and where

I felt I could make a difference and it

and I realized um I studied math and I

was okay at math but I know I I have

known mathematicians who are really

talented gifted mathematicians the one

who really make a difference I wasn't

going to be one of those people likewise

playing music I don't have that

intrinsic Talent it's fun I can play

songs in front of people and do stuff I

like it and stuff like that but I don't

have that kind of talent in fact I'll

say something that that I say to friends

sometimes and you're a good friend of

mine if I had the talent to get a few th

people on their feet dancing by playing

music I'd probably just do that really

as long as we've been friends I knew

none of this I like knew none of this

mostly because I think we always end up

talking about Neuroscience or other

aspects of our life um but I I didn't

know I I know I know a great many things

about you but I had no idea it's so

interesting um do you still do dance we

had Eric Jarvis on the podcast by the

way Professor Rockefeller who studies um

at one point was studying um uh Speech

in so in Birds um and song in Birds and

then he's done a great many other things

now in genetics of of vocalization and

you know he actually uh danced with or

was um about to dance with the Alvin ay

Dance Company it's like a really really

talented dancer um uh and so you know

dance seems to be like a theme that