As others have said, the midsized cities are going to be your best bet. Ulm was lovely when I was there, incredibly charming especially in the fall when I was there.

Of course there are many types of German cities, so Ulm will show you something different than, say, Lübeck or Bremen. Southern Germany is what most Americans associate with Germany though, which Ulm can give you.