Just to reiterate - it isn't that surveillance is illegal. It is that surveillance of people who have given no cause for suspicion is.
Otherwise, what the heck is the fourth amendment for?
Targetted surveillance of folks about whom there is reasonable suspicion looks permissible.
Broad surveillance is precisely the thing the fourth amendment rules out.
And creating records about everyone for subsequent use is clearly an element of surveillance under any reasonable definition.
If such records existed for some other purpose, maybe their storage would be justifiable. But actually, they have no other purpose. And therefore they are held explicitly for surveillance uses. Pretending otherwise would be sophistry.
The key reason for the secrecy of the programme has surely been that the US government knew their judgement of what is lawful would not bear scrutiny.