Plans to build houses at a former aluminium factory in Pontarddulais have taken a step forward, although Swansea Council highways officers said it would lead to delays for motorists in the area.