To prevent your beer from becoming infected and tasting terrible, cleaning your carboy is very important. When you have emptied a carboy of fermented beer there often is a ring around the top where the foam has touched. This will require a brush such as this one.
Fill the carboy with a water and sanitizer solution following the directions for the sanitizer. PBW is a good sanitizer and does not require a rinse after cleaning. A couple scrubs from the brush and the residue should be gone. Empty out the carboy in the sink or tub and place a sanitized stopper and airlock in the top. This should keep it fairly clean for when you are ready to use it next. You should always rinse the carboy with sanitizer before using it to make beer. You should never use a cleaner such as bleach or soap as it can effect the taste and quality of the beer.
PBW is a great cleaner, but it’s not technically a sanitizer. It won’t kill bacteria like a sanitizer will. It just gets the “gunk” off the carboy (and very well, at that).
Sanitizers are solutions like Star-San and Iodephor. These should be used right before using it to make beer, like you say. This will kill all the bacteria.