Why not filter out the files that you don't need to rename?
Summat like:

(Get-ChildItem -Path "path\to\folder" -Recurse | Where-Object {$_.Name -contains '•'} | Rename-Item -NewName {$_.Name -replace '•',''} -verbose -ErrorAction SilentlyContinue -ErrorVariable daError)