Another tip from Ellenbogen's Kitchen - .NET Postback in modal dialog.
If you’re trying to postback from a modal dialog window, you’ll see that a new window will open up with the results. In order to stay at the same window, use <base> element inside the document’s <head> element –
<html>
<head>
….
<base target=“_self”/>
</head>
….
</html>