Initially used as an analgesic and anesthetic, ketamine has unfortunately been abused as a popular recreational party drug due to its psychotropic effects. Over the last decade, ketamine has also emerged as an effective rapid-onset anti-depressant. The increasingly widespread use and misuse of the drug in infants and pregnant women has posed a concern about the neurotoxicity of ketamine to the immature brains of developing fetuses and children. In this review, we summarize recent research findings on major possible mechanisms of perinatal ketamine-induced neurotoxicity. We also briefly summarize the neuroprotective effects of ketamine in the presence of noxious stimuli. Future actions include implementation of more drug abuse education and prevention campaigns to raise the public’s awareness of the harmful effects of ketamine abuse; further investigations to justify the clinical use of ketamine as analgesic, anesthetic and anti-depressant; and further studies to develop alternatives to ketamine or treatments that can alleviate the detrimental effects of ketamine use, especially in infants and pregnant women.