Would me bringing my router upstairs and hooking it up directly to my computer give me better internet connection, opposed to having it be wireless over two floors?
Definitely, having the data transfer right there to you from the router as opposed to it traveling across open space is much faster. Depending on how you test the result, you may not even know the difference unless prior to that, your wireless signal was really weak.