Here's the thing I don't like about the government looking into my private affairs.
Many years ago, when I went into the Army, I needed a Top Secret Crypto clearance to be able to do my job. In order to get that, I had to have a background investigation. Before that happened, I had to acknowledge that I knew that was a condition of my being able to get the clearance and be trained in the specialty I had chosen, and sign off on paperwork giving my consent for the BI.
Some months later, while I was home on leave, I had people I had known for years - schoolteachers, classmates, even relatives - ask me what kind of trouble I was in. When I asked what they were talking about, they mentioned visits by men with FBI badges, asking questions about me. When asked why they were asking about me, they wouldn't say why, which led folks to believe I might be in some sort of trouble.
Flash forward to now. I have a job in the transportation industry, where I have to have access to government facilities from time to time; specifically, to the local Marine Terminal, where military equipment and weapons are gathered, stored and shipped overseas. It is none of the government's business what overseas calls I make, or what web sites I visit, or what books I buy or check out at the library, or what my interests run to aside from my job. But what do you think my boss would do if some FBI agent showed up at my job, asking questions about me and refusing to say why, based on information the government gathered illegally about my phone contacts, reading habits, and web browsing habits?
That's why they are supposed to get a warrant. They are supposed to have to go to a judge and show just cause why they should be able to invade my privacy and gather that information. They are supposed to have a compelling reason why that should be allowed, some proof that there is some sort of activity going on that they need to investigate - not just some guy who has friends overseas, in the Middle East, from his travels there over the years; not just some guy who has always had an interest in the subjects I mentioned earlier, and checks out books in the library or buys books or visits web sites to pursue his interest in those subjects.