Do you have an old PC lying around gathering dust? How about a small-capacity USB flash drive sitting, unloved in a drawer? You can reuse your old computer and a USB flash drive by installing a tiny Linux distribution.
Mini Linux distros are great as they require fewer system resources than other options yet still deliver a whole operating system experience, and we have nine of the smallest Linux distros for you to choose from.
I was impressed with antiX as a light-weight system. If you are ok with a tiling window manager, ArchBang is good too.
They are essentially stripped-down Debian and Arch respectively.