We often think about how the foods we eat affect our hearts, waistlines, and even the environment. But they also have a major impact on our brains.

As a nutritional psychiatrist, I study how dietary changes can lead to meaningful improvements in our mood and mental well-being.

Foods like leafy greens, colorful fruits and vegetables, seafood, and nuts, beans and seeds all promote brain health.

But below are five foods to avoid if you want to keep your memory and focus sharp. Of course, it's hard to eliminate these from your diet completely, so moderation is key.

1. Foods made with industrial and processed seed oils

Highly processed oils are often extracted from soybeans, corn, rapeseed (the source of canola oil), cottonseed, sunflower and safflower seeds, and contain a lot of omega-6 fatty acids.

Excess consumption of omega-6s can trigger the body to produce chemicals that can lead to inflammation in the brain.

If you're sautéing vegetables or grilling fish or meat, I recommend using olive, coconut or avocado oil.

2. Foods with added and refined sugars

Our brain uses energy in the form of glucose, a type of sugar, to fuel cellular activities. But a high sugar diet can lead to excess glucose in the brain.

This can cause memory impairments and less plasticity of the hippocampus, the part of the brain that controls memory. 

Don't forget that many savory foods have hidden added sugars, too, like store-bought pasta sauces, ketchup, salad dressings and even canned soups. Swap these out for homemade items made with whole foods.

3. Processed foods

A diet high in ultra-processed foods may put you at risk of having shorter telomeres — or the "cap" on our DNA. Longer telomeres tend to promote healthy cellular aging. Shortening our telomeres may mean that we are at risk of degenerative disease earlier in life.

A 2022 study also found that participants who consumed high amounts of ultra-processed foods like baked goods and sodas were more likely to experience mild depression compared to those who consumed the least.

Here's a tip: If you can't pronounce an ingredient, or have no idea what it is, it's often best to avoid it.

4. Foods with artificial sweeteners

When you use artificial sweeteners that have no nutritional value, they can increase "bad" gut bacteria which can negatively affect your mood.

These sweeteners include saccharin, sucralose and stevia. Aspartame can be especially harmful, and has been directly linked with anxiety in research studies. It also causes oxidation, which increases harmful free radicals in the brain.

Some alternatives to consider: Honey, monk fruit extract or coconut sugar.

5. Fried foods

While items that are battered, crusted or deep-fried may be at the top of the comfort foods list, they can be damaging to the brain.

A study of over 18,000 people found that a diet high in fried foods was linked to lower scores of memory and cognition.

As an alternative, I suggest opting for baked, air-fried, or steamed versions of your favorite foods.

Dr. Uma Naidoo is a nutritional psychiatrist, brain expert, and faculty member at Harvard Medical School. She is also the Director of Nutritional & Lifestyle Psychiatry at Massachusetts General Hospital and author of the best-selling book "This Is Your Brain on Food: An Indispensable Guide to the Surprising Foods that Fight Depression, Anxiety, PTSD, OCD, ADHD, and More." Follow her on Twitter and Instagram.

Don't miss:

• A Harvard nutritionist shares the No. 1 vitamin that keeps her brain 'young and healthy'—and what to eat
• This is the 'worst food ingredient for your immune system,' says immunologist
• Nutritionist shares her top 5 ‘healthy’ fast food orders—they include Taco Bell, Chick-fil-A and Panda Express

Want to earn more and work less? Register for the free CNBC Make It: Your Money virtual event on Dec. 13 at 12 p.m. ET.

Sign up now: Get smarter about your money and career with our weekly newsletter