Yes, you go to the office and ask her out to lunch the next time you see her.
If you don’t do this, you’ll just end up doing what you’re doing right now.
You will just keep overthinking this until you either talk yourself out of it or lose whatever opportunity you may or may not have with her.
You are scared because you want to be 100 percent certain that she likes you.
This is why asking her out to lunch is to your benefit.
Remember, you are not asking her out to be your girlfriend.
You are just asking her out to lunch as a work colleague.
Coworkers ask each other out to lunch all the time.
It is normal.
You have also already had several break room lunches with her in the past.
So you have already built a degree of rapport with her, thereby making it even more likely that she will either say yes to your request or at least consider it.
Getting her outside of the work environment will give you the best chance at loosening her up.
This is where it will be easier for you to discover whether she’s seeing someone else or if she’s actually available.