You can set different date custom fields for each calendar. Go to Calendar's Cog/Set Up and on the main page choose Start Date Custom field and End Date Custom Field. Here only existing date or date time custom fields can be selected from dropdown menu.

By default, the fields are filled in with Two Custom Fields selected in Calendar's configuration.