I’m not completely sure I understand what you are describing, but I’d guess it’s either
(a) BIOS/UEFI setup setting that is telling the system to boot USB drive; and the fix is changing this setting on your machine so it boots your hdd/sdd first
(b) you didn’t install the bootloader, or installed it on the wrong device (such as your maybe your install thumb-drive?). What I mean here is shown at the bottom of this install picture from the Lubuntu manual
though if you chose other options during install it may have appeared elsewhere on the screen.
source: https://manual.lubuntu.me/1/1.3/installation.html
If it’s (b) the steps I’d use to fix can be found at Grub2/Installing - Community Help Wiki which need to be adapted to your machine, so a re-install may be faster/easier if it looks complex.