As the conference gets under way, and I’d like to set some context for your sensors &

electronics under the title ADAS & Autonomous Systems – a sense of reality

1
2
I hope the UK examples will not confuse as we still drive on the correct side of the
road, because Napoleon never got round to changing it.
I will rush through the scenarios. You may not have time to take them in …
but you will have more time than if you were driving through them.

3
So is there a problem with AVs?
Since that unfortunate Google Lexus had a fender-bender with a bus, there has been
a change in media tone as anticipated by this slide from June 2015. And YouTube is
thick with instances where Tesla drivers have misunderstood AutoPilot limitations.

When AV cedes control on out-of-scope state,

driver the is not ready

Such condition will demand higher skill

level than normal
So maybe we cannot get to full autonomy via levels 3 and 4. It’s 5 or nothing, or a lot
less

4
We hear figures for the safety potential of AVs – 93% being commonly used as the
reduction possible by eliminating the human driver;

5
Let’s look behind these headline figures at the driver failures.
In a highly detailed causation study by IFFSTAR in France, loss of control failures
accounted for about 20% of causations, while 30% were in-control errors of guidance
hitting infrastructure, and around half were in-control errors of guidance colliding
with other vehicles. Behind these categories is an important cascade, split by driver
functional failure causes, where half comprised detection of hazards, around 15%
evaluation of what was actually detected, then if those steps were OK, 8% failed to
make sense of the situation, 10% understood it but made the wrong decision, and
about 15% didn’t manage to pull the right manoeuver off.

6
As can be seen from the PSA-Renault Lab for Accidentology and Biomechanics
analysis for the RoadSense project , these functional failure category ratios are
related to accident configuration, too detailed to study now, but we can discuss
afterwards. But just for an example, we can see on the right the driver functional
failure categories, with a split by collision scenario along the front, perception –failure
to see- at junctions is the worst failure.

7
Another caution comes from official causation data from the UK police …
Here, actual human error contribution is not 93%, it is 39%. Even when all driver
performance contributory factors are included, the driver is responsible for only ¾ of
cases. Road, vehicle, visual interference account for around one eighth, and over 10%
were blamed on pedestrians.

8
Coming back to the oft-quoted (mis-quoted) USA NHTSA National Motor Vehicle
Crash Causation Survey, practically all moving vehicles have a driver on-board, and it
is difficult to think of crashes where at least one driver is not involved. In 95% of
cases (weighted to represent national accident configurations) the driver was
responsible for the critical reason for the critical event that made a crash completely
unavoidable; in other words the drivers failed to extricate the vehicle from the point
of no return. But what got them there in the first place? In 43% the drivers were
involved in the hazard becoming a make-or-break scenario. Road and weather
accounted for 30% and vehicle deficiencies for 16%.
Taking the driver out of the loop will not help as much as is claimed!
Vehicle shortcomings will still be present – advanced systems must never succumb to
neglect, and must never be foiled by infrastructure and climate.

9
But whichever way you look at it, the problem is simple – to keep at least one of
these conditions from arising in conjunction with the other two.
The solution is more complex.

10
Let’s look at the now widely accepted collision timeline in detail -
Normal traffic exists until a potential hazard is encountered; attention paid to this
may confirm there is a hazard. At this stage, the hazard is avoidable, but without
appropriate action, a collision is inevitable, even if its severity can be modified by late
action.
This is followed by an impact of the vehicle against the hazard object (assuming no
pedestrian/cyclist involvement foe this illustrative example)and within milliseconds
the human occupants impact the vehicle interior, whether benignly as in the case of
an airbag or belt, or injuriously againt a hard component.
Somebody makes an emergency call, or it is automatically initiated, the emergency
services respond, and the recovery team remove the wreckage while police or other
accident investigators commence their study, concluding later with a report.

11
And in close-up, SAE levels 2, 3 ad 4 largely concentrate on averting collisions just
before or just after they become inevitable, which is an area bordering on the
behaviour of a risky driver, inhabiting the zone where potential and actual hazards are
generated, and must be reacted against.
Safe experienced drivers display greater observational skills, and have an awareness
of the small clues to hazards and adapt to avoid them while still making swift
progress – some of the best at doing this that I know are also bikers and have learnt
in depth – those that I still know. Inhabiting the critical re-crash envelope, ADAS and
risky drivers are actors in the pre-crash event and its prequel, the motion before the
Pre-Crash Event. Inhabiting the critical re-crash envelope, ADAS and risky drivers are
actors in the pre-crash event and its prequel, the motion before the Pre-Crash Event.
So lets look at a couple of examples …

12
and now a skilled driver reacting to an escalating hazard, the result of an unusual
road layout:

13
and now a skilled driver reacting to an escalating hazard, the result of a poorly-
marked, unusual road layout: the yellow van makes a last-minute change of direction,
realising he is in the wrong lane for a right turn in an interlaced, offset cross-roads,
while the Porsche driver moves right to give the van more room.

14
Post-script: Within days of this recording, possibly by coincidence, the road markings
were renewed for several miles of this route.

15
In the Swedish application of the Traffic Conflicts Technique, the volume of fatal
accidents is small compared to the volume of serious ones which are in turn small in
relation to moderate cases, and so on through to those with no injury,

16
But these are just the collisions, and thus only a tiny volumetric proportion of the
conflicts where at least one party has to take evasive action to avoid a collision, and
so on until the vast bulk of traffic proceeds with no disturbance. At every boundary
level, the differences between avoidance and escalation can be a very small, random
difference in any one of the contributory factors. Understanding these transitions is
key to good driving as well as good traffic management.

17
18
Returning to sensors and systems, then – what are we asking of them, and what are
the issues that can impede their performance in the real world?

You will recognise these headings as no different from the human failings found
earlier.
Let’s look at some examples.

19
[run and read]

20
At this point the driver of the camera car could see first see the cyclist, approximately
340m distant, about twice as far away as the oncoming car

21
Approximately 55m further on, car is clearly visible, cyclist is not visible to camera nor
to pedestrian detection software, (temporary “No Overtaking” sign visible on right.
Due to loose gravel and no road markings following re-surfacing)

22
Approximately 62m further on, car has passed, cyclist is not visible to camera nor to
pedestrian detection software, it would now appear to be safe to overtake if there
was a slower vehicle in front, and no temporary ban

23
62m further on, the cyclist is discernible, but would be at high risk from an overtaking
vehicle by this point.

24
Even with a 1920x1080 camera, if we zoom in, we do not see what the human saw
• top left, just to left of white sign
• top right, somewhat to right of white sign
• bottom left, range about 130m not clearly detectable optically, possibly with radar
or LIDAR
• Bottom right, sufficiently clear to be detected and classified - very late if doing a
passing manoeuvre

25
Here, low sun disables Forward Collision Warning, Lane Keep Assist and Traffic Sign
Recognition system – and blames the driver for a dirty screen – as you can see it was
fine for the human driver, and our forward scene camera.

26
One could accept the system creating that message in these circumstances
But as with this human driver, full control of direction must still be maintained
despite a braking from 61 to 52 mph on perceiving, evaluating, and interpreting the
hazard, and deciding the action, the water then caused a drop to 33 with the brakes
released, the traction and stability control only intervening as the driver started to
resume cruise speed.

27
So, evaluation
The arrow on the road says this way, the speed sign says no more than 10, but is it a
No Entry?

28
These need to be understood …

29
As do these…

30
And these …

31
And these white lines – err no, street car rails, which err, don’t mean a thing as to
these guys even in the absence of road markings

32
And these should not be mis-interpreted – they are common in Europe showing the
vehicle speed limits for various categories of road, in this case in km/h filmed on a UK
motorway where we use miles per hour.

The TSR system under test does not interpret these as road signs ….

33
But on a clear stretch of road, for no known reason it displayed a 10 mph limit while
legally doing 70 on the M1. Fortunately this is a TSR system, not a camera-based ISA,
and posed no hazard to following traffic.

34
And again it came up with 90 which is 20 more than anywhere in the UK – and in the
absence of any foreign trucks.

35
This sign shows a legal maximum speed limit of 30 mile/h, but in fact the three
diagonal bars mean it starts in another 300m (1500 ft),
but if we look 50m further down the road …

36
There is a smaller 50 mile/h limit repeater sign, this is the controversial county-wide
rural road speed limit imposed for this class of road, at great expense and the police
declined to enforce (but don’t rely on that!)
And another 50m down the road …

37
We have the same 30 mile/h sign, this time a 200m warning. Although maybe
helpful, these signs are explicitly illegal according to the UK Department for
Transport’s “Traffic Signs Manual”. It is possible that with a black circle instead of red
that they were permitted but that was withdrawn before these signs were erected.
Either way, you won’t find them in any catalog when setting up your Traffic Sign
Recognition System.
This raises concerns for Camera-based Intelligent Speed Adaptation!

38
But this system under test is doesn’t know that.

39
Here we have a post for the black-diagonal-on-white-circle sign for no special speed
limit (which means national 60 for this class of road as you all should know) also
carrying a legal 40 sign on the same pole, while the black circled sins mean only
advisory – except don’t tell anyone I said so because that’s not in the official catalog,

40
In this example, three shapes can be extracted which initially conform with
pedestrian image proportions. On closer approach, it can be seen that two are postal
installations and the third is a slightly larger-than-life image of a woman in an
advertising poster. Quite how these would be rejected, if at all, is dependent on the
sensing technology.
Interestingly, there is a real pedestrian, red-circled in the second image, who emerged
from a car parked off the road at a lower level and does not conform with pedestrian
image proportions, but in this case he is not at hazard anyway.

41
A pedestrian is captured here walking between slow-moving traffic on a motorway…
In fact there are two of them …
But they are just images on the back of a van. Again, false positive is possible with a
camers/image analysis system, but unlikely with a camera plus ranging technology

42
Some people never learn - Here are two grown men, playing around in higher speed
traffic at night …
– hopefully deep learning is better

43
Not quite like the last two examples, here is a real man behind a van, but
– target shape keeps changing
and
- is not in “correct” proportion

44
Those were isolated examples, but in real life more things happen at once.
Lets throw everything at it:
uncontrolled single lane contraflow traffic due to un-mapped construction,
application of “priority to traffic on your right rule” abandoned in the name of
common sense,
the red van meets a vehicle in the single-file section, and will have to reverse
but waits for us,
but we have to wait for pedestrians in dark clothing approaching from behind
in the dark.
.

45
CAVT happened to be first-on-scene at this incident which occurred in darkness after
light rain, leaving the road surface damp. The case vehicle is in the field, visible at
night as its lights were still on and the occupants were still emerging ; these photos
were taken the next day. A number of previous crashes, mostly single vehicle
running off road had observed here previously, and some evidence of them can be
seen here, including missing important chevron bend warning signs, and tyre tracks,
…

46
A slippery road warning sign flat on the ground, over more old tracks, but beyond
where muddy tracks on the road – a little too late to be of use. Another crash site
can be seen on the top right, …

47
That was either an over-correction and run-off-road travelling in the same direction,
or in the opposite direction – cars had been seen in the ditch for both options
previously!

48
Back to our case – at first sight, the vehicle appeared to have left the road here
entered the ditch, and rolled diagonally the travelled some distance airborne to its
final position, indicating rather a high speed after leaving the round. However the
debris found in this trail does not match the case vehicle and this interpretation was
ruled out. The tracks further on proved more interesting.

49
Here the damage trail, debris and vehicle damage are consistent with several
unsuccessful attempts to stabilise the car after an initial action to avoid an oncoming
car partially on the wrong side back where the mud was, ending with the car leaving
the road yawing about 30 degrees left into the ditch and rotating further as it
mounted the hedge and rolling 360 degrees to the right before landing on its wheels.
This is also consistent with the accounts of the driver and three occupants given to
the investigator and their statements to the police at the scene. ( Injuries appeared to
be slight; the occupants had walked through the field to the gate where the
investigator had pulled in, without the opportunity to view the trail.

50
Looking back to the gate, still on the morning after, …

51
… here the mud can be seen carried onto the road by agricultural vehicles, again
consistent with the driver’s account.

52
So eventually the highway authority spent the money, not for the benefit of AVs, but
to give humans a fighting chance.
Obtaining the money to upgrade the roads like that to make them suitable for
autonomous vehicles, ADAS, or just humans, has to come from somewhere, and
seems to only be found after persistent known dangers, even fatalities,

53
The situation in the USA doesn’t look hopeful either.
Incidentally, bringing roads up to standards necessary for AVs is also going to make
human drivers much safer too …
… eroding the case for AVs?

54
55
56
57