Blaming retailers for sweatshop conditions and promising to punish the wicked among them, the garment workers union escalated its fight in the city yesterday. More than 400 garment workers marched down lower Broadway in the sweltering heat, chanting "No more sweatshops" as SoHo shopkeepers scurried curbside to see the spectacle. The union targeted Ann Taylor, Eddie Bauer and Gap Inc.'s Banana Republic chain, accusing each retailer of fostering oppressive labor conditions in the manufacturing of their clothing.