Sometimes You Have to Fire Yourself: The Evolution of an Active Stock Investor Turned Passive

Michael QuanBeliefs, Education, FI / FIRE, Investment, Misc, Saving Money9 Comments

Some links below may be from sponsors. Please see our disclosure for more info.

It took me 20 years, but I eventually did fire myself from active stock investing.

For the record, I didn’t have terrible performance, nor did I get ensnared in an SEC insider’s trading probe.

So, why in the world would I fire myself?

Shifting Perspective

With age, comes perspective and when I finally hit 40. It put some things into perspective.

Thankfully, my 40s is a lot better than I once had imagined.  I’m actually a lot healthier now than I’ve been in the past decade, and just as excited to watch life unfold… even after 4.5 years of FIRE.

Life moves pretty seamlessly from one decade to the next, but somewhere along the path, you realize that your perspectives have shifted.

We simply value different things at different stages of life.  My readers over the age of 40 can attest to this, right?

In my 20s I was driven by earning a lot of money and having fun.  In my 30s, I dumped all of my efforts into my career building a business.  And now in my 40s, my focus has shifted to my family and how I can help others.

The Gambler’s Itch

When I was in elementary school, I remember taking many trips to Las Vegas.  My Uncle (who FIRE’d in his late 30s) had a motor home that could fit both his family and ours.  So, we’d make semi-annual trips to Las Vegas on long weekends or the summertime.

I LOVED Las Vegas. The lights, sounds, and excitement that radiated through the city were intoxicating.  I seriously could not wait until I turned 21 so that I too could gamble in hopes of winning a HUGE jackpot.

This passion persisted up into my early 20s before it was suddenly shattered.  You see, I read this book – American Mensa Guide To Casino Gambling which gave me a rude statistical awakening about how much the odds favored the house!  I was devastated because I realized that gambling would never be my ticket to FIRE.

With the gambling bug out of my system even before I turned 21, I focused my sights on the stock market!

Stock Market Riches

Finally, I found a place where I could turn my seed money into rooms full of gold!  After reading book after book, I thought I couldn’t lose.  Every data sheet said the stock market returned 9-10% annually over the course of many decades.

But wait, 9-10%?  That doesn’t sound like that much.  Surely I could beat that, I thought.

And so my adventure as an active stock investor began…


In my last year of college, I opened up my first brokerage account with $500.  We were still in the dot-com boom, so I invested in CSCO (Cisco – which makes routers that power the Internet).  I also bought some DELL, EBAY, and MACR (Macromedia – web publishing software).   Even with the boom of the markets back then, a shift in a share or two within these companies didn’t really do much.

Maybe I should day trade on margin? I pondered.

Thankfully, I never took this idea too far because I ended up landing my first job out of college soon thereafter.

[Hindsight 20/20: I’m really glad I dove into the market, even if it was just with a few hundred dollars.  It got my “feet wet” and it gave me some momentum investing.]

First Job

The company had a 401k plan and I decided to max it out.  Outside of that, I decided to save and also invest via ShareBuilder (a dollar cost averaging brokerage house).

Instead of choosing funds, I was a big believer of finding the RIGHT companies.  I even picked up some AMZN (Amazon) for $10!

[Hindsight 20/20: I was definitely smart to begin investing right away.  I received the company match and started building some equity into the 401k funds as well as my ShareBuilder account.  However, my stock selection process was a bit rudimentary. 

I would typically look at the Morningstar ratings and went with tech companies because that’s what I knew at the time.  I had no idea about index funds back then, what an asset class was, nor any associated risk!  I did continue educating myself though, which is how I eventually learned to dollar cost average.]

My Company

Once I started my own company, we eventually implemented some retirement plans for the partners, and eventually the employees.  The employee plan included a matching component and it was set up by an investment advisor from Edward Jones.

During this decade I continued investing in both my Sharebuilder and IRA accounts.  I also began learning about real estate investing during this time.

It was also during this span of time when I became a gold bug!  I listened to many different doomsday podcasts which forecasted a spike in silver, gold, and oil.  And, for a while, they were right.

Buy Buy Buy!!! After my gold bug years, I also started to watch Jim Cramer on CNBC.  I even purchased his newsletter for a year and tried to mirror his selections.  I learned a little bit about stock pricing watching him, and I also adopted the belief that I could selectively time the market.

[Hindsight 20/20:  It was eye-opening to see how many of my employees didn’t contribute the minimum to our investment plan.  I also realized many years later just how many fees we were paying.  We had a choice of several different American Funds and a lot of them had both front & backend loads (commissions)!  When I finally figured out* how much they were siphoning off if fees, I yanked all the money into a rollover IRA and invested in low-cost index funds.

I probably re-allocated a good 25-30% of my portfolio into commodities during my gold bug years.  It actually did okay for a while, but when the economy came roaring back a few years later, all of those gains were wiped out and I re-allocated back to blue-chip stocks.

While it was fun to watch Cramer on Mad Money, I ended up chasing my tail a bit.  I definitely didn’t beat the market using his strategies.  I realize now just how tough it is to figure out WHAT to listen too when you’re first starting out.]

*The two sources I credit for exposing me to all the hidden fees were a book (MONEY Master the Game: 7 Simple Steps to Financial Freedom) and Personal Capital’s Fee Analyzer.  It saved me hundreds of thousands of dollars in fat fees!

The FIRE Years

When I finally exited the workplace in 2013, we had a portfolio of $430,000 in equities.  This was divided amongst my Sharebuilder, Roth IRAs, Rollover IRAs, and misc scattered accounts.

I continued to invest in my Sharebuilder account for a while but eventually moved it to my Etrade account because Sharebuilder at the time couldn’t title the account into the name of our trust.  I never did restart the auto-investing because Etrade didn’t have the type of functionality.

In early retirement, my strategy was preserving capital and maximizing gains.  It was around this time I signed up with Personal Capital and began tracking our personal financials on their platform.  I noticed that our accounts were gaining some decent ground, but it always trailed the S&P 500 Index by ~1%.

[Hindsight 20/20: Investing in your FIRE years without a substantial income is a whole new game.  While I was able to grow our assets, I’m quite confident that luck during this incredibly long bull market was a factor.  I also stopped investing our Roth IRA contributions the past 3 years.  I would make the contribution and then just let it sit in cash.  The main reason because I would get caught up in analysis paralysis.

It turns out, it’s statistically better to just invest a lump sum rather than dollar cost average the lump in overtime.  I won’t get into the details here, but this book The 5 Mistakes Every Investor Makes and How to Avoid Them: Getting Investing Right was a huge factor in helping me to understand my personal biases and irrational thoughts.  Let’s just say my eyes were opened!  (Kind of like when I realized gambling was a losing proposition.)]

You Don’t Know, What You Don’t Know

So, as the saying goes, you simply don’t know what you don’t know.  It’s a strange realization, but rather enlightening.  And just knowing this has yet again shifted my perspective.

If I look back on the past 20 years of stock investing, there was A LOT I didn’t know.  So, how could I have performed better?

Well, I could have found an Independent Advisor (IA) who was a full fiduciary.

The IA could have helped me to invest in low-cost index funds much earlier on.

The IA could have also helped me to tax plan and minimize my tax exposure (i.e. think tax harvesting, or backdoor Roth conversions).

Finally, an IA could have saved me a TON of time.  I’ve spent countless hours trying to “beat” the market unsuccessfully.

Evolution: Getting Out of My Own Way

Are you seeing a clearer picture yet?  It was time to get out of my own way, my own biases, and automate this baby!

This is why I gave up active investing in stocks and have handed our portfolios over to a specialized IA*.

I still don’t know what I don’t know, but I know that and I have a strategy to leverage smarter people than me to handle it.

My IA isn’t just one person, rather a team of specialized individuals:

  • Certified Financial Planners
  • Certified Public Accounts
  • Attorneys

So there you have it.

I’m getting out of my own way and tapping the expertise of a team.  However, I don’t consider my past two decades a waste at all.  I learned a ton, and have enough financial knowledge to ensure my IA team is consistent with our agreed upon strategy.

This is going to cost me 1% consistently, but I believe it will be made up in tax, legal savings, etc… not to mention I was already giving up a full point by underperforming.

*Finding the right IA is CRITICAL before hiring them… I’ll leave have to leave that to a future post, and again Peter’s book can help – The 5 Mistakes Every Investor Makes and How to Avoid Them: Getting Investing Right.

The Next Phase

I’m pretty happy with my decision.  There’s no longer any pressure for me to perform and keep such a vigilant eye on things.  I’m still in equities for the long game and don’t plan to touch the capital base for many decades to come.

I suppose you could call me an official boglehead now.  I’ve also resolved that I’m not the next Warren Buffet, so I may as well take his direct advice on investing.

I will still stay curious and follow the results of those active stock investors who seemed to have figured it out, but I’m going to be a spectator this time around.

Final Thoughts

Sometimes in life, we need to realize we can benefit from something (i.e. the stock market, technology, etc.) without getting our hands too dirty.  And, that’s a wonderful thing!

I think I’ve still got a little of the gambler’s itch deep down inside, but instead of trying to beat the market, I’m going to focus on other areas that are more impactful like building a business, investing in real estate, and being present with my family.

Remember, reaching FIRE doesn’t need to be complicated.  It just needs to be consistent with the right strategy (check out my book, The F.I.R.E. Planner for more strategies to achieve FIRE).

Readers, do you have a clear investment strategy when it comes to stocks?  How has it changed over time?  How do you mitigate the risks of what you “don’t know”?

Follow me

9 Comments on “Sometimes You Have to Fire Yourself: The Evolution of an Active Stock Investor Turned Passive”

  1. It’s just a lot simpler to be a passive investor. You just let your money sit, forget about it (and genuinely and truly forget about it, not like when people suddenly turn into “long term investors” as soon as the market dips), and move on.

    It just gives you more time to focus on other things in your life.

  2. Yeah, I realized this myself this week. I haven’t ever been an active investor, I was so passive that I didn’t bother to even see what my financial advisor was really doing. So I made mistakes in that regard, but then I went the other way. Managing my own investments and thinking about things way too much. But I realize now, that I value simplicity. Simplicity of my investments and simplicity in the amount of decisions I need to make. Parenting is already hard enough. I killed off my play portfolio this week and I immediately knew it my gut it was the right decision. Glad to see you are moving onto the next phase too. Cheers!

    1. Yes, simplicity is key! Although it’s expensive, I’m enjoying not having to manage my portfolio directly. This could change in the future but I’d still remain passive… at least for the bulk of our equities.

  3. I remember having a Sharebuilder account years ago. It was one of the few where you could buy fractional shares on a scheduled weekly basis! If only I had held on to some of those stocks I was buying, which leads straight to your point here. During that time I was also contributing to my Roth IRA, putting almost all of the funds in VFINX and then later switching to VTSAX(after reading some of JL Collins posts). It wasn’t as exciting as picking individual stocks, but looking at my accounts now I am very satisfied with the returns. I still pick stocks here and there for fun, but only with a very small percentage of my portfolio. But, even doing that, it can be hard to not go overboard and get FOMO.

    1. Yeah, Sharebuilder was great back then!

      FB, you must be quite satisfied with your long-term returns. Nice work keying in on VFINX and VTSAX early on.

      What stocks do you like for your “fun” account?

      1. I hold some MSFT, AAPL, and TSLA. And on the far more speculative side I’ve recently picked up a couple of EV startups – ARVL and CCIV(soon to be Lucid). Not doing so hot at the moment, but planning to hold and see where it goes! Hence the reason I only play with a small percentage and the bulk of everything is in index funds.

  4. I’ve thrown a few speculative bets out there for the thrill of the chase, but mostly growth names that are hurting my overall return at the moment haha. This year has definitely reinforced my VTSAX plan.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.