Let's say you have two relatively complicated networks at two remote sites, one behind a FortiGate firewall and one behind a pfSense firewall, like this:
Site A
Firewall: FortiGate
LAN1: 172.16.0.0/24
LAN2: 172.16.1.0/24
LAN3: 172.16.2.0/24

Site B
Firewall: pfSense
LAN1: 172.17.0.0/24
LAN2: 172.17.1.0/24
LAN3: 172.17.2.0/24

You want to set up things so the routing between the two sites is as simple as possible while still allowing for stuff to change in future, things like one site adding more networks and so on.

Here's the trick... Anyone familiar with the way FortiGate does IPsec will know about the 0.0.0.0/0.0.0.0 way of setting up a P2 selector. You do that and then use static routes and IPv4 policies to determine what actually goes over the tunnel. Now the pfSense world doesn't quite do the same (don't try it, you will mess up your firewall), but there is a similar type of thing called a Routed Mode tunnel (VTI) that actually inter-operates with the FortiGate way of doing things quite well. VTI isn't entirely complete yet, but it works.

P1
We won't discuss your P1 setup here, the real stuff happens in the P2. Just set your P1 up in the same way as usual, with careful negotiation and agreement between the network admins on both ends.

P2 on the FortiGate
FortiGate will try to do the P2 in the right mode right out of the blocks, you basically just need to agree the Encryption related stuff with the other end. You leave Local Address and Remote Address both as 0.0.0.0/0.0.0.0.

Once you have the tunnel defined you need to get set up static routes to the remote networks. Just define three routes to 172.17.0.0/16, 172.17.1.0/24 and 172.17.2.0/24 on the new tunnel's interface.

P2 on pfSense
This is where it gets interesting... You want to create a P2 with settings like this:

Mode: Routed (VTI)
Local Network: 172.31.255.1/30 (or any other private IP space /30 that is unlikely to be used on either side. Note the "network address" is actually the first host IP in the /30)
Remote Network: 172.31.255.2 (basically the second host IP in the /30, adjust to taste)
The rest of the stuff... as agreed with the FortiGate end

Now you have your P2 defined, go to Interfaces - Assignments and drop down the list in the last row. There should be a new interface called something like ipsec1000, add that interface and then open its configuration. Tick the Enable box and give it a more meaningful name than OPT1. Don't change anything else. Theoretically, once this is applied, your tunnel should come up and do... well nothing really.

Next go into System - Routing and notice the shiny new gateway that was created when you enabled the new interface. Go into Static Routes and define routes to the 172.16.0.0/14, 172.16.1.0/24 and 172.16.2.0/24 networks via your new gateway.

And there we are
Congratulations! You now have routing between all 6 networks set up and if your firewall defaults are sensible you should see... well no connectivity between the networks at all. Now you can go do clever things like setting up your firewall rules (IPv4 Policies in FortiGate language) to allow stuff that needs to be allowed.

One important point to remember about VTI tunnels... Your firewall rules that relate to stuff coming from the networks at Site A must go into the IPsec ruleset, even if a ruleset for your new tunnel interface appears (it didn't on my setup, so it may only be on older versions of pfSense where this confusing ruleset appears).