Exchange 5.5 does not support rejecting invalid recipients at the SMTP transaction level.
If you are using Exchange 2000, you should look at third party tools to reject invalid messages at the SMTP level.
Both GFI Mail Essentials and Vamsoft's ORF can filter on the active directory.
Exchange 2003 introduced a new facility to filter email messages if the email address doesn't exist in the Active Directory. This will stop spammers from sending messages to non-valid addresses. The rejection is done at the SMTP level - so the email message isn't even delivered.
Enabling the Option on Exchange 2003
1) Expand ESM, Message Delivery.
2) Right click on "Message Delivery" and choose Properties.
3) Click on the tab "Recipient Filtering".
4) Enable the option "Filter Recipients who are not in the directory."
You then need to enable the Recipient Filter on the SMTP Server.
1) Still in ESM, Expand Admin Groups, , Server, , Protocols, SMTP.
2) Right click on SMTP Virtual Server and choose Properties.
3) Click on "Advanced" next to the IP address on the first tab.
4) With the IP address selected, choose "Edit".
5) Enable "Apply Recipient Filter".
6) Click Apply/OK until clear.
The ability for Exchange 2007 to drop email for unknown recipients is available as part of the anti-spam functionality, but is not enabled by default. The management control for anti-spam is only installed by default on the Edge role, for other roles (for example on a single server deployment) the anti-spam functionality needs to be installed by running a script.
Anti-spam Controls Installation.
1) In The Exchange Management Shell, change to the following directory (presuming default installation location) C:\Program Files\Microsoft\Exchange Server\Scripts
2) Use tab to select install-AntispamAgents.ps1 and press enter.
3) Once it has completed, restart the Exchange Management Console and go in to Organisation Configuration, Hub Transport and select the Anti-spam tab. If you cannot see the Anti-Spam tab then close the console and restart the Microsoft Exchange Transport Service.
Enable Recipient Filtering.
1) All of the anti-spam options are enabled by default. For now, disable them all except for recipient filtering.
2) Go in the the properties of recipient filtering and enable the option "Block messages sent to recipients not in the global address list" on the Blocked Recipients list. Apply/OK out.
3) Restart the Microsoft Exchange Transport Service again.