First, you might call the Inland Wetlands Office (203-245-5632) or visit the Land Use Office in the Town Hall at 8 Campus Drive, to look at maps of your property to see how the wetlands are mapped.
Second, you might talk with the Inland Wetlands Enforcement Officer about what is "Regulated" by the Inland Wetland Regulations as a "Regulated Activity." A "Regulated Activity" means any operation within or use affecting a wetland or watercourse by obstruction, by construction, by alteration, by removal or deposition of material or by pollution of such wetlands, whether or not they appear on the Official Inland Wetlands and Watercourses Map of the Town of Madison.
Some activities require an Inland Wetland Boundary Clarification to be approved by the Inland Wetland Agency, before you can proceed with obtaining a Regulated Activity Permit.
Some regulated activities can be approved by the Inland Wetlands Enforcement Officer, and others are approved by the Inland Wetlands Agency.