不用動腦動手——自動化時代的隱患

Robert Siegel （Host）： And I’m Robert Siegel， on the road for a talk with author Nicholas Carr. Carr’s books are the 1）nagging， 2）tech-wary 3）conscience of the digital age. In The Shallows， he warned that surfing the Internet is destroying our 4）attention span. And now in his new book， The Glass Cage， Nicholas Carr warns us that computers are starting more and more to make decisions for us and that we risk forgetting how to make those decisions ourselves. He writes a lot about cars. So we are going to pick him up in this 5）state-of-the-art， high-tech driving machine—a 2014 Mercedes-Benz S550 4Matic 6）sedan. It’s a 7）loaner， and Michael Minielly of Mercedes-Benz USA is our 8）chaperone. Thank you very much for lending us this car this morning.

Michael Minielly： Absolutely. Happy to be here.

Siegel： And since this car so far out of my league—talking about a hundred-twenty-thousand-dollar car here？ Minielly： Just about that. Yeah.

Siegel： You’re going to have to explain to me all the amazing things this car can do for me without me having to do them.

Minielly： Absolutely.

（Soundbite of car door closing）

Siegel： This is Mercedes’ 9）top-of-the-line， highest tech model car you can drive. It parks itself. It controls the 10）windshield wipers. Driving in the dark？

Minielly： So we have a system called active 11）high-beam assist， which 12）automatically 13）illuminate the road in front of you.

Siegel： And if a car’s approaching in the opposite direction？

Minielly： It will automatically lower them and continuously lower them to the amount where it’s—it will never 14）dazzle the driver.

Siegel： Talk about dazzling the driver—this car has something called Distronic Plus.

Minielly： There’s the 15）stereo camera， and then there’s also short-， medium- and long-range radars that are continuously monitoring the traffic situation， both in front and behind the vehicle.

Siegel： At one point， as I was driving in rush hour traffic to Nicholas Carr’s hotel in Washington D.C.， I tried this Distronic Plus system out. The system kept me in my lane. Had I 16）strayed， it would’ve taken over the steering. And it maintained my distance from the taxi that had just cut in front. The car ahead of me is moving， so...

Minielly： So you don’t have to use your...

Siegel： The car is following it.

Minielly： Yeah.

Siegel： I’m not 17）accelerating right now.

Minielly： That’s correct. And you’re not braking.

Siegel： I’m not braking. And now the car ahead of me is slowing down， so this car’s slowing down.

Minielly： Correct.

Siegel： No hands， no feet—the Mercedes was driving itself. Nick Carr， welcome.

Nicholas Carr： Thank you.

Siegel： I decided to pick you up in the All Things Considered mobile studio.

Carr： （Laughter） It’s very impressive.

Siegel： （Laughter） I have arrived driving Nick Carr’s nightmare. We set out for a glimpse of what’s to come in nearby Arlington， Virginia. And as I drove， we talked about automation and what it has done for us and to us.

（Soundbite of Global Positioning System）

Unidentified Woman： In 700 feet， turn right onto New Hampshire Avenue Northwest.

Siegel： For Nick Carr， automatic 18）navigation demonstrates how technology gives to human beings while also taking away.

Carr： At least you used to have to figure out where you were. And even with a paper map， you’d have to locate yourself somewhere and figure out what the landmarks around you are and kind of get a sense of place. And that’s no longer necessary，when you have， you know， the voice come on and say， in 500 yards， turn left—200 yards， turn right. I do think there’s something lost there.

Siegel： And it’s not just behind the wheel of a vehicle that can drive itself where Nicholas Carr sees a worrying loss of 19）autonomy.

Carr： Well， you see it in a lot of professions. So if you go in these days to have your annual physical， usually now the doctor’s spending a good deal of time looking into a computer to enter information that they used to dictate—notes about the visit—or write down themselves—to kind of go through processes， 20）templates that give them hints on diagnosis and stuff. Can be good—can be not so good， but changes the doctor-patient relationship in very interesting ways. Pilots—the name of the book The Glass Cage refers to what pilots call the glass cockpit—that more and more， they’re flying by looking at banks of computer monitors.

Siegel： But you acknowledge that， overall， the number of air crashes we’ve had over the past couple of decades since we’ve automated is way， way down.

Carr： It is. I mean， the whole history of 21）aviation is a history of flight becoming much， much safer. And some of that is definitely due to autopilots and automation systems. But I think what we’re seeing—and， you know， the Federal Aviation Administration is concerned about this and has sent out alerts—is that pilots just aren’t getting enough practice in 22）manual flying. Almost the entire flight now is done on autopilot. And so on those rare—thankfully， rare occasions when something unexpected and bad happens， pilots often make mistakes because their skills have gotten 23）rusty. So I think the lesson isn’t that automation is bad， but we have to be very wise in knowing how to automate and when to say， no， let’s not take more control away from the human being.

Siegel： I’ve been talking with Nicholas Carr， whose new book， The Glass Cage， is a—is a warning against the potential dangers of some kinds of automation. And so far， we’ve been in the present—at least， a very expensive version of the present—driving this amazing Mercedes-Benz. But now we’re at the Virginia Tech Transportation Institute， at their research center in Arlington， Virginia. And now we’re going to get a glimpse of the future with Ray Resendes. Tell us about the Cadillac we’re about to take a ride in.

Ray Resebdes： So we have a Cadillac SRX. We have added a technology called dedicated short-range communications. Coupled with GPS， the vehicle puts out a basic safety message， which， basically put， is saying， here I am. Here’s the location of a vehicle. Other cars equipped with the same technology can hear that and just like an air traffic control system， can start figuring out who’s in my path. Am I in an 24）imminent crash danger？

Siegel： Virginia Tech and the Virginia Department of Transportation have also placed short-range radio 25）transmitters on selected roads and highways. Ray Resendes’ Cadillac receives their 26）transmissions and displays them on the 27）dashboard navigation screen. We see a recommended speed on an 28）exit ramp. As we pass a school， we get a warning that schoolchildren are nearby. When a second Virginia Tech car outfitted like the Cadillac passes us， we get an alert. Nicholas Carr says he worries about people being bombarded with such automatic alerts in our future cars and everywhere else.

Carr： This is something called alert fatigue. Doctors now—when they use computers， they’ll often be alerted to possible drug 29）interaction problems. These alerts come up so often now that the doctors start just 30）dismissing them automatically because a lot of them are completely useless， it turns out. There is a big problem that the systems become so sensitive and give you so many alerts that you actually become less alert yourself because you’re just dismissing the alerts.

Siegel： And here’s another question that Nicholas Carr writes about. If the car of the future will make decisions for us， how will it decide what to do when a 31）collision is unavoidable， and a computer is in charge of the steering？

Carr： You have to start programming difficult 32）moral 33）ethical decisions into the car. You know， if you are gonna crash into something， what do you crash into？ You know， do you go off the road and crash into a 34）telephone pole rather than hitting a 35）pedestrian？ Or—you know， things that we do—and we might not do them that well all the time， but...

Resebdes： Very interesting answer ’cause if you look at crashes， the best crash to get into is to rear-end somebody else ’cause cars are well-designed to deal with that. A lot of times， people will steer to avoid a 36）rear end collision. And then you can run into 37）head-on collision， which is the worst crash， or you can hit a pedestrian. Having to 38）codify these issues into the 39）algorithm on a vehicle is a—is a very serious issue.

Siegel： That’s Ray Resendes of Virginia Tech. Nicholas Carr’s complaint against 40）intrusive automation isn’t just about how well or how poorly computers might make moral decisions for us. It’s about the very 41）erosion of human autonomy.

Carr： Once we start taking our moral thinking and moral decision-making away from us and putting it into the hands not of a machine， really， but of the programmers of that machine， then I think we’re starting to give up something essential to what it means to be a human being.

Siegel： That’s Nicholas Carr， author of The Glass Cage： Automation and Us. By the way， he told me that he drives an Audi A4 with no navigation system.

羅伯特·西格爾（主持人）：我是羅伯特·西格爾，我正在去采訪作家尼古拉斯·卡爾的路上。卡爾的書是有關這個令人對科技產生不安與警覺的數字時代。在《淺薄》一書中，他提醒我們上網會影響我們注意力的廣度。如今在他的新書——《玻璃箱》中，尼古拉斯·卡爾提醒我們計算機開始為我們作越來越多的決定，我們承擔著遺忘如何自己作決定的風險。他寫了許多有關汽車方面的問題。所以我們將要用這臺最先進的、高科技的駕駛機器——2014年出的型號為S550的四輪驅動奔馳轎車去接他。這輛車是借來的，美國奔馳公司的麥克爾·米尼利是我們的陪同人員。非常感謝（貴公司）今早能把車借給我們。

麥克爾·米尼利：應該的。非常高興能來到這里。

西格爾：因為這輛車讓我望塵莫及——它應該價值12萬美元吧？

米尼利：差不多吧，嗯。

西格爾：不用我來說，你會給我解釋這輛車能為我做的所有神奇的事情的。米尼利：當然了。

（車門關閉的聲音）

西格爾：這是奔馳最高科技的頂級轎車。它能自行停車、操控雨刮器。如果在夜間行駛呢？

米尼利：我們有一個叫活躍遠光燈輔助的系統，它能自動為你的前路照明。

西格爾：如果有一輛車從相反的方向駛來呢？

米尼利：那么這個系統就會自動把光調暗、持續把光調暗到一定程度——它從不會讓司機感到眼花。西格爾：說到讓司機感到眼花——這輛車有一個增強型限距控制系統。

米尼利：車上裝有立體照相機，在前后都裝有短程、中程、遠程的雷達持續掌握交通狀況。

西格爾：在某段時間，我正在高峰期開車去尼古拉斯·卡爾在華盛頓下榻的酒店，我試了下這個增強型限距控制系統。這個系統讓我能保持在一條車道上。如果我偏離了車道，它會接手控制方向。它能保持我與剛才前面突然插進來的的士之間的距離。前面的車在移動，所以……

米尼利：所以你不需要用你的……

西格爾：這輛車在跟著它。

米尼利：嗯。

西格爾：我現在沒有加速。

米尼利：沒錯。你也沒有剎車。

西格爾：我沒有剎車。現在我前面的車在逐漸減速，所以這輛車也開始逐漸減速。

米尼利：沒錯。

西格爾：沒有用手，沒有用腳——這輛奔馳能自動駕駛。尼克·卡爾，歡迎你。尼古拉斯·凱爾：謝謝。

西格爾：我決定在《面面俱到》的移動演播室里迎接你。卡爾：（笑）這真讓人印象深刻。

西格爾：（笑）我開著尼克·凱爾的“噩夢”而來。我們

出發去看看弗吉尼亞的阿林頓周邊的情況。我邊開車，邊和他討論著自動化以及它給我們所帶來的影響。（全球定位系統的聲音）不知名女士：前方700英尺，向右進入新罕布什爾州大道西北段。

西格爾：在尼克·卡爾看來，自動駕駛展現了科技為人們帶來的利弊。卡爾：至少過去你需要找出你在哪兒。就算有一張地圖，你也需要了解自己的位置，找出周圍的標志性建筑，有一種位置感。（如今）這些都不需要了，當你有需要，你懂的，一個聲音就會出現：前方500碼，轉左；前方200碼，轉右。我真的認為那樣會有問題。

西格爾：能自動駕駛的汽車只是尼古拉斯·卡爾擔心人們丟失自主能力的一個方面。

卡爾：好吧，你知道自動化在各行各業都有滲透。因此如果你要去進行每年一次的體檢，現在的醫生通常都會花很多時間去查看他們過去在電腦里記錄或者自己寫下來的信息——比如門診筆記。反正就是一種走程序的形式，按照模板來進行診斷。這樣做有可能好也有可能壞，但卻非常有趣地改變了醫生與患者間的關系。飛行員——那本叫《玻璃箱》的書說的是飛行員們口中的玻璃座艙——飛行員越來越依賴一排排的計算機顯示器來進行飛行。

西格爾：但是你得承認，自從我們進行自動化飛行以后，在過去的好幾十年里總的來說墜機事件的數量在直線下降。

卡爾：確實是。我的意思是，整個航空史就是讓飛行越來越安全的歷史。這其中一部分的原因當然是因為采用了自動駕駛儀和自動化的系統。但是我認為我們所看到的是——你知道的，美國聯邦航空管理局所擔心并且發出警示的是——飛行員沒有獲得足夠手動駕駛飛機的訓練。現在幾乎所有的航行都由自動駕駛儀來完成。所以在有些時候，當一些意想不到的糟糕情況發生時，飛行員通常都會犯錯誤，因為他們的技能已經不夠嫻熟，而慶幸的是這種情況很少發生。所以我想從中得出的教訓不是自動化不好，而是我們必須明智地知道什么時候應該使用自動化，什么時候不應該使用，我們不要過多地剝奪了人類的自主控制。

西格爾：我一直在和尼古拉斯·卡爾聊天，他的新書《玻璃箱》提醒我們一些自動化使用的潛在危機。目前為止，我們還在現實中，至少是一個非常昂貴的現實——開著那么棒的奔馳。但是現在，我們身處弗吉尼亞大學交通學院所在阿林頓的研究中心，我們要與雷·雷森迪一道放眼未來。跟我們說說這臺即將載我們去兜風的凱迪拉克吧。

雷·雷森迪：那么我們有一臺型號為SPX的凱迪拉克。現在我們在車內加入了一種叫專用短程通信的科技，與全球定位系統一道，這輛車可以發出一種基本的安全信息。大致上就是能說：“我在這兒，這里是車的所在位置。”同樣擁有這一科技的其他車輛就能接收到信息，就像空中交通控制系統一樣，我就能開始知道誰跟自己同道。我即將會有發生碰撞的危險嗎？

西格爾：弗吉尼亞理工大學以及弗吉尼亞的交通部門也在選擇的路段和公路中裝上短程的無線電廣播發射機。雷·雷森迪的凱迪拉克接收到了他們傳送的信息，并在儀表板導航屏幕上顯示出來。我們在經過出口匝道時看到了建議時速；當我們經過一所學校時，我們收到了附近有學生出行的提醒。當另外一輛跟這輛凱迪拉克有同樣配置的弗吉尼亞理工的轎車經過我們時，我們收到了警示。尼古拉斯·卡爾說他擔心未來的轎車或其他每個地方都會受到自動警報的轟炸。

卡爾：這叫作警惕疲勞。現在的醫生——當他們在使用計算機的時候，會經常被提醒藥物交互作用可能產生的問題。這些提醒出現得太頻繁，以至于醫生都自動忽略它們，因為許多警示都是毫無作用的。這里有一個很大的問題，這個系統變得如此敏感，它給了你那么多的警示以至于你自己實際上會變得沒那么敏感，因為你一直在忽略那些警示。

西格爾：尼古拉斯·卡爾還寫到了另外一個問題。如果未來的汽車能為我們作決定，當計算機操控著駕駛時，那么當碰撞不可避免時它將如何作出決定？

卡爾：這樣你就必須在車內編寫復雜的倫理道德決定程序。你懂的，如果你將要撞到某個東西，該撞什么呢？你懂的，你會開到路肩以外而撞向那個電線桿而不是撞向一個路人嗎？或者，你懂的，我們做的事——我們不可能總是能做得那么好，但是……

雷森迪：這個答案很有趣。因為如果你研究車輛的碰撞問題，就會發現最理想的碰撞就是追尾，因為汽車的設計能很好地處理這個問題。很多時候，人們都能避免追尾的碰撞。但是你也有可能遇到正面的碰撞，這就是最可怕的，或者你可能會撞到一個行人。把這些問題編成計算程序運用到汽車上是一件很值得認真思考的事。西格爾：這是弗吉尼亞理工大學的雷·雷森迪。尼古拉斯·卡爾對侵入性自動化的抱怨不僅僅是關于計算機能給我們作出多好或者多壞的道德決定，而是關于人類自主權被侵蝕的問題。

卡爾：一旦我們開始剝奪我們的道德思維和道德決策，沒有把它交給機器，真的，而是交到編程人員的手中時，那么我想我們就已經開始放棄對人類來說至關重要的東西了。

西格爾：這是尼古拉斯·卡爾，他是《玻璃箱：自動化與我們》的作者。順便說一句，他告訴我他現在開的是一輛沒有導航系統的奧迪A4。