From Conversations With Tyler: Jack Clark on AI’s Uneven Impact:
COWEN: Do you end up with too much surveillance over your kid? You’ll know everything if you want to, right? That might be a reason why you give the kid the bunny all the more.
…
COWEN: Say you have the hypothesis that so many seven-year-olds — they talk to themselves. They say weird things, and the AI bunny is going to report back to you, “Your kid says — ”
CLARK: If the bunny says your kid is saying strange stuff —
COWEN: But they all say strange stuff. I might have been chattering on about the New York Mets at age seven, and it would have been perfectly harmless, but it may not have sounded that way.
CLARK: You’re going to need to create spaces for unmonitored creativity in people. Actually, the same as how we have an approach to AI research today. Now, AI systems can output their chains of thought, which is the reasoning they use to come up with answers. We’ve had this question at Anthropic of how much should we monitor the chains of thought?
If you actually monitor them, you might create an anti-goal where the system ends up wanting to have chains of thought which are safe to be monitored and which don’t cause demerits, which might actually break how it thinks in a bunch of ways. I think this analog applies here, where you’re going to need to choose how much you actually decide to know about people, or you risk creating incentives that change their behavior such that they have a negative effect.
Starting ≈13:38.