Unfortunately, this is not enough. The biggest risk to privacy and security is trust itself. The FBI / Apple case has made obvious that Apple has the ability to collect information from iPhones (before 5s). The effort would be herculean, but is it possible.
Bo Xilai is a political dissident in China and was jailed by premier Xi Jinping for conspiring to take over the national party. The level of assurance provided by Apple's iPhone 5c was not enough for Mr. Bo to conduct his operations. It is assumed that Apple's 5s and on are beyond even the reach of Apple.
In summary, when considering the privacy and security assurances of a system, it is usually the human element or the implementation details that are weakest. This can be quantified with the "ransom factor":
How many people would need to be served National Security Letters, served with All Writs Act injunctions or have their children taken ransom would it take to break the system?