If sufficiently outnumbered, defenders may surrender immediately. If you do not want to assault it directly, you will have to wait until the residents surrender, usually from starvation (30 days in a castle and 100 days in a town).
The siege equipment constructed is unique to the fortification and takes time based on your engineering skill to build. However, to assault a fortification requires construction of ladders or siege towers. No special equipment is required for laying siege.
