You should probably adjust the settings on your device, The New Oil has a good guide:
In terms of digital assistants Siri is the least invasive, most audio of your requests to Siri are processed on device using the Apple Neural Engine. Ask Siri and Dictation sometimes sends your requests to Apple, for instance if you ask Siri to play a song in Apple Music Siri will need to contact Apple’s servers to find the song and play it. You can test this though, try asking Siri something without any internet connection. According to Apple requests that are processed on Apple’s servers are associated with a random identifier which is rotated on a 15 minute basis.
From Apple’s Privacy Policy:
Apple stores transcripts of your interactions with Siri and may review a subset of these transcripts. Siri may also send information like your voice input, Siri setup, contacts, and location to Apple to process your request. Data is not associated with your Apple ID.
On-device intelligence such as the suggestions that Siri does of your apps is done on your device and is synced across your devices using end-to-end encryption through iCloud. However some information is sent to Apple about the personalisation but it isn’t associated with you.
It’s up to you to decide whether the helpfulness of Siri is worth keeping on. I would also have a read of these privacy pages on Apple’s website so you can better decide.