Police said they were called to Grazebrook Croft in the Bartley Green suburb of the city at 11.20am on Friday. The victim, who is believed to be in his early 50s, sadly died at the scene.
West Midlands Police said they do not know why the man was attacked and searches are underway for the culprit.
Detectives have been carrying out door to door enquiries in the neighbourhood and scouring CCTV for any clues.
A police drone has also been capturing images to help with the probe.
Specialist search teams have also joined forensic experts as they search for evidence.
Grazebook Croft and neary Bigwood Drive were closed by West Midlands Police whose spokesperson said they will remain cordoned off until at least Saturday.
He added that people living in the roads will be allowed into or out of their homes, but they will be disrupted and may need to wait to be escorted though the area.
Officers were working to identify the victims so they could tell his family the sad news.
READ MORE ABOUT A LIST OF ‘PROBABLE’ NUCLEAR TARGETS IN THE UK