Yup, that's the idea. Pretty much ANY 100mbit or gigabit ethernet switch will do the job, so just get the cheapest one. Avoid "hubs" (different from switches, even though they look alike).

I don't know if it's applicable here or not, but sometimes running an ethernet cable is impractical or hazardous (eg. across an open floor or hallway). For such situations, "powerline" adapters are very handy.

The idea is they begin as a basic pair of devices. One device is plugged directly to a wall receptacle near your router, and a short ethernet cable is used to connect it to the router or switch. The matching device can then, generally, be placed anywhere in the residence, again by plugging directly into a wall receptacle near the desired location. The house's AC power wiring acts as a "network cable" between the two devices.

That second device can then provide a local ethernet jack to run a short cable to a computer or wireless access point, or it could itself be a wireless access point. Eg:

https://www.amazon.ca/TP-Link-ADVANCED-Universal-Powerline-TL-WPA4220/dp/B00HSQAIQU/

The more basic models (without the built-in Wifi in the second device) are trivial to install: just plug them in, and press a button on each of them within a minute or two of each other, and they will self-detect and configure.

EDIT: Once one has a matched pair of such items working, you could purchase additional ones from the same model line (same manufacturer) to add more remote ethernet jacks, one by one (no need to use pairs after the first two).