Narrow optimization can lead to unintended consequences
Information scientist Steve Omohundro shows "that rational systems exhibit universal ‘drives’ towards self-preservation, replication, resource acquisition and efficiency and that those drives will lead to anti-social and dangerous behaviour if not explicitly countered."