you need to parenth around your 2 OR parts.
Also if your joins are not set you may be getting a cartesian result that is over loading the report
{APM_MASTER__INVOICE_PAYMENT.Bank_Account} = {?Bank Account} and
{APM_MASTER__CHECK.Check_Date} = {?Check Date Range} and
(
{AP_Invoice.Approved_Amount} <>
{APM_MASTER__INVOICE_PAYMENT.Amount_Paid} or
{AP_Invoice.Approved_Vendor_Number} <> {APM_MASTER__INVOICE_PAYMENT.Vendor}
)