Monday, December 21, 2020

The conjecture that the energy of a static electric field is zero solves the infamous 4/3 problem of classical electrodynamics

This Physics Stack Exchange post (2013) describes the 4/3 problem:


Richard Feynman in his Lectures on Physics (1964) defines the energy density of the electromagnetic field as

      u = ε_0 / 2  E^2  + ε_0 c^2 / 2   B^2.


One can derive from the definition of u that the Poynting vector of energy flow is:

       S = ε_0 c^2 E × B.


The energy flow agrees with what we empirically observe in electromagnetic radiation.

                    |  -
       ----------------------
         |    |    |    |      E
       ----------------------
                    |  +

                          ---->
                          v

However, for a static electric field, S gives nonsensical results. Imagine a capacitor where the electric field lines are as indicated.

If we move the capacitor to the right at a speed v, the magnetic field vector B stands up from the diagram toward the reader of this blog. The Poynting vector S correctly says that there is energy flow to the right in the space between the capacitors. 

Near the capacitor plates or the edges, the electric field is more complicated and we do not claim anything about the energy flow there.

But if we move the capacitor upward, then E and v are parallel, B is zero, and S is zero. The Poynting vector claims that there is no energy flow in the space between the capacitors! The energy flow is different for an upward movement than a sideways movement. That defies our intuition about how mass-energy moves in space.

The energy flow happens in a complex way close to the plates and at the edges of the capacitor, and probably that flow transfers the energy up when we move the capacitor upward.

The 4/3 problem concerns the electric field of a charged sphere when it is moved. Also in that case, we get a nonsensical result. The Poynting vector gives a larger momentum for the field than what we get from the expression of u above.


A static field has zero energy


The problems disappear if we define the energy density of a static electric field as zero. The energy density E and B of electromagnetic radiation we define in the old way.

Do we always know what field is static and what field is radiation? If a charge is accelerated, then part of the field is static and part is radiation.

The "sharp hammer" model, which we introduced a couple of days ago, helps us to distinguish what is the static field component and what is radiation. The point charge repeatedly applies the Green's function to create the electric field. If the charge moves at a constant velocity, then there is a total destructive interference of energy-carrying photons. But if the charge accelerates, then the destructive interference is not complete. There is energy in the field.

Thus, if a charge moves at a constant velocity then its field has zero energy. If it accelerates, then part of its field does contain energy.

Electromagnetic waves have no static field. Their whole field contains energy.

Our rubber plate model of the electric field helps to understand all this. If the charge is moving at a constant velocity, then there is no stretching in the plate and zero energy. If the charge accelerates, then the charge makes a "hill" in the rubber plate. There is now energy in the field because stretching requires energy.

If the charge moves back and forth, it creates waves in the rubber plate. Those waves carry energy far away - they are electromagnetic waves.

The choice of u and S is not unique. Does there exist a choice of u which would be non-zero for a static electric field, but for which S would give sensible results?

Divergence in the vertex correction in QED

NOTE December 22, 2020: 
Zoltan Harman (2014) writes that the ultraviolet divergence is only in the case where the momentum q of the incoming photon is zero.

There is no ultraviolet divergence if q != 0.

The case q = 0, of course, can be reduced to the electron self-energy case, which we handled by banning self-energy loops altogether (the center of mass argument).

There is an infrared divergence. That is handled by setting a little mass m to the photon carrying the momentum k.

-----

In the past two weeks we have argued that the Feynman integral in QED for vacuum polarization converges (but not absolutely), and the electron self-energy loop can be removed altogether.

The third divergence in QED is in the vertex correction:

                     photon
                     4-momentum k
                     ~~~~~
                   /               \
          e- ------------------------
                        /
             ~~~~
          photon
          4-momentum q

The divergence is logarithmic.

For example, the electron may scatter from the field of a nucleus Z+. Then the incoming photon carries spatial momentum q.

Our rubber plate model of the electric field of the electron says that the plate will bend when the electron accelerates toward the nucleus. The plate resists the acceleration.

When the electron recedes from the nucleus, the rubber plate tries to keep it going. The electron emitted a photon which contains spatial momentum k, to the rubber plate, and then absorbed the same photon.

Thus, in the classical world, the electron does send an off-shell photon to itself in the encounter with the nucleus.

If the electron exchanges momentum q with the nucleus, we can calculate the rough geometry of the encounter. 

We have the principle that virtual photons only live for about 0.1 times their (Compton) wavelength. From the geometry of the rendezvous we can then set a cutoff for possible momenta k. The divergence is removed.

Alternatively, we may use destructive interference. If the approaching electron would send a short-wavelenth off-shell photon (momentum k) to itself, there would be lots of destructive interference at the point where it absorbs the photon. We would get an extra coefficient 1 / |k| to the Feynman integrand, which would remove the divergence.

The famous anomalous magnetic moment of the electron is calculated using many diagrams with the vertex correction. We need to check if our cutoff procedure would somehow alter the calculation. The calculation agrees with the experimental value at a precision 10^-10. We do not want to change that.

Note that in the vertex correction, we could have very heavy particles with large charges doing the rendezvous. The calculation has to agree with the classical limit. We need to verify that is the case.

In classical electrodynamics, the magnetic field resists changes in the current. A single electron can be regarded as a current. The rubber plate in this case seems to do the work of the magnetic field: it resists changes in the velocity vector of the electron.

Our rubber plate model tries to solve the long-standing problem of the interaction of a charge with its own field. The vertex correction is a prime example of such self-interaction.

Sunday, December 20, 2020

A running coupling constant breaks the classical limit

There is nothing which prevents the particles in a Feynman diagram from being extremely heavy, making them essentially classical particles. We could have 1 kg particles doing Coulomb scattering.


In formula (63), Matthew Schwartz (2012) presents the vacuum polarization effect on a spatial momentum transfer. The effective coupling constant becomes smaller when -p^2 is increased:

       α_eff = 1 / 137 (1 +  0.00077 ln (-p^2 / m^2),

where m is a fixed energy, for example, m might be 511 keV.

Let us then imagine that we have very heavy particles with very large electric charges. They are the classical limit of Coulomb scattering. We can monitor the paths of the particles very accurately.

Suppose that we then double the mass and the electric charge of our particles. The formula above claims that the coupling constant changes, because the momentum transfer p is double. The change is not small, about 0.1 %.

That does not make sense, since we are working at the classical limit. The Coulomb force should have the same formula for all classical objects.

What is wrong?

Our new way of handling vacuum polarization restores the classical limit.

Recall that a virtual pair typically lives just for 0.1 Compton wavelengths. Here, we define the generalized Compton wavelength for a general 4-momentum as

        λ = h c / |k|,

where k is a 4-momentum (E, p), and we define

      |k| = sqrt(E^2 + p^2).

The norm || is the euclidean length of the 4-momentum vector. It is not the Minkowski metric length sqrt(-E^2 + p^2).

Let the heavy particles pass each other at a distance L. We add to the usual Feynman vacuum polarization integral an extra coefficient

         C(|k|) = 0.1 λ / L = 0.1 h c / (|k| L),

if C(|k|) < 1. If it is bigger than 1, we do not add the coefficient.

That is, we only take into account those pairs which are born at a distance < 0.1 λ from the passing particle.

The coefficient C(|k|) makes the Feynman integral to converge. Or does it? We have to find online a brute force calculation of the Feynman integral with a cutoff |k| < Λ and check that it only diverges logarithmically on Λ.


L. Alvarez-Gaume and M. A. Vazquez-Mozo (2010) have done the calculation (though they did not include it in their paper).

Their formula (372) states:

        Π(q^2) = e^2 / (12 π^2)  log(q^2 / Λ^2)
                         + finite terms.

There, the integral has been taken over all |k| < Λ. The integral diverges only logarithmically when we increase the (large) cutoff  Λ.

Our corrective term C(|k|) makes the integral to converge, because it has 1 / |k| in it. But we are also interested in a numerical value.

Suppose that heavy particles pass each other at the distance of 1 meter and exchange momentum q. How does our corrective term C(|k|) affect the value of the integral?

The value of |q| is macroscopic. Its Compton wavelength is microscopic, of the order 10^-34 m. If |k| is of the order |q|, then our corrective term is roughly 10^-34.

In the SI unit system, the 4-momentum of the electron at rest is ~ 10^-21. Our corrective term for it would be ~ 10^-13.

The integrand in formula (365) of the paper for |k| << 10^-21 is roughly

        1 / |q|^2,

which is integrated over a 4-volume 10^-84. The value is negligible.

We conclude that the integral is essentially zero when we correct the integrand with our term C(|k|).

The effect of vacuum polarization is essentially zero when macroscopic charges pass each other at a distance 1 meter.

We have restored the classical limit. Heavy particles behave as in the simplest Feynman diagram for Coulomb scattering. Vacuum polarization, which would depend on q, has essentially a zero effect.

Apparently, prior to this blog entry, no one has noticed the classical limit problem in Coulomb scattering vacuum polarization. Breaking the classical limit is a serious error in quantum mechanics.

The rubber plate model of the electromagnetic field explains momentum conservation

         ^
         |
         |
        ●                     ●
        +                      -

Assume that we have opposite charges initially static, like in the diagram. They pull on each other.

Let us then use a rocket to accelerate the positive charge upward very fast, so that it attains almost the speed of light.

The negative charge on the right still sees the positive charge at its original location for a while. The negative charge gains a momentum p to the left during that time.

We assume retardation. Signals do not travel faster than light. The negative charge does not know that the positive charge has moved.

Meanwhile, the positive charge gains a momentum q right, but |q| < |p| because the positive charge rapidly moves away.

Where did the difference of those momenta go? Could it be that the negative charge somehow emitted photons to the right, and those photons took the difference? That looks very implausible. The accelerating positive charge does emit energy, but that happens symmetrically and cannot take the extra momentum.

How can we ensure that momentum is conserved?

The rubber plate model comes to the rescue. The negative charge on the right pulls on the rubber plate of the positive charge. The negative charge gains a momentum p left, and the rubber plate gains a momentum p right.

The rubber plate delivers the momentum p later to the positive charge.

How can a massless rubber plate store momentum? Where is its inertia? Again, it is the finite speed of light which mimics inertia. When the negative charge pulls, the elastic  rubber plate stretches a little bit. The longitudinal stretch wave travels at the speed of light to the positive charge and tugs on it.

We need to check if anyone has proved conservation of momentum under retarded forces. Gravity and the Coulomb force should honor conservation.


The Wheeler-Feynman (1949) absorber theory tries to tackle this issue.


S. Carlip (1999) writes that in General Relativity, a binary system conserves angular momentum (ignoring gravitational radiation). Retardation of forces would violate angular momentum conservation, but the effect is canceled by other factors.

Laplace in 1805 calculated that the "speed of newtonian gravity" has to be at least 7 million times the speed of light, so that retardation effects do not make the orbit of the Moon unstable.

Apparently, momentum conservation in classical electromagnetism is an open problem.

Saturday, December 19, 2020

The energy of the static electric field of the electron is zero, not infinite?

Our "sharp hammer" hypothesis from last week suggests that the energy of a static Coulomb field is zero! The field consists exclusively of virtual photons which contain spatial momentum but no energy. Why would the field then contain any energy of its own at all?

The sharp hammer hypothesis is that the static Coulomb electric field consists of the Green's function of the electromagnetic field applied at the location of the point charge at infinitesimal time intervals.


                  | knock knock knock
                  |
                  v
               #      
               #=======   sharp hammer
               v
  _______________ drum skin
               ●
              "point charge"

The classical analogue is that we hammer a drum skin at the same location at very short time intervals. A permanent, time-independent, depression is created in the drum skin. It is like a static Coulomb field.

In the classical case, there is energy stored in the static deformation of drum skin.

Classically, in newtonian mechanics, we can explain the movements of charges without assuming any energy stored in the electric field. It is just the particles that exert forces on each other.

Maybe the right quantum field model for the static electric field of an electron is that it is a pure spatial momentum field that does not contain any energy at all?

Energy is only present in Fourier components where the 4-momentum k contains energy. Since for a static field, those components have a total destructive interference, there is no energy in the static electric field.

This would solve the paradox which has been present since the discovery of the electron at the start of the 20th century: how can a point charge have a finite mass even though the energy of its static electric field is infinite?


An early attempt to "renormalize" the electron mass


A simple attempt to resolve the paradox was to assume that the electric field is cut off at some very short distance dr from the electron, and the mass of the "bare" electron has a large negative value -M. When we add -M to the very large energy E of the electric field, we get:

       E - M = 511 keV,

that is, the well-known electron mass visible to the outside world.

The solution to the paradox is often viewed as an early example of renormalization, where we assume that the "bare" mass (or some other property of a particle) has a wildly different value from the "dressed" mass visible to the outside world.

The notion of the electron having a negative bare mass is strange. Negative masses behave in a strange way in classical physics. 

Suppose that the renormalization scheme is correct. Then we could collide the electron with another at a very high energy. The electric field of the electron cannot react to the collision instantaneously because the speed of light is finite. The collision might reveal that the bare mass of the electron is -M, a negative value. That would be a strange observation.


The self-energy diagram of the electron


                            photon which the electron
                            sends to itself,
                            4-momentum k
                                ~~~~~
                             /                \
 electron e-   -------------------------
 4-momentum p
                            A                B

The "self-energy" diagram of the electron is depicted above. The Feynman integral for the diagram contains the product of:

1.  the electron propagator (4-momentum p - k) from A to B and

2. the photon propagator (4-momentum k) from A to B.

The integral diverges when we integrate over all 4-momenta k.

The standard way to solve the divergence problem is to put a cutoff to k and assume some "bare mass" -M for the electron, such that the electron mass visible to outside world becomes m = 511 keV.

However, our considerations about the conservation of the speed of the center of mass suggest that the diagram above is forbidden, or that the photon must be absorbed in zero time, and the the effect of the diagram is then actually null.
               
              real photon, energy E
               ~~~~~~~~>
             /
       e- ● 
        <-

Let us analyze the diagram using the particle model. We may assume that the electron is initially static.

Suppose that the pointlike electron emits a real photon to the right, but after some finite time interval dt, reabsorbs it. During the time interval dt, the electron moves left.

The real photon carries some energy E to the right at the speed of light and is partially responsible for keeping the center of mass of the system static.

But then a miracle happens: the electron magically captures back the energy E carried by the photon. The electron is again static, but it has moved a little bit to the left. The center of mass has moved.

Could it be that at the time of absorption, the electron somehow jumps to the right to its original position? It might be, but in the momentum phase wave description of the process there is no such sudden jump in anything. When an electron absorbs a photon in the wave model, the electron does not suddenly jump to another place. How could the electron even know where it has to jump when it absorbs a photon?

What is going on here? If we think of classical particles, they only absorb other particles which are at their current location. They do not magically absorb particles which are far away. Working in momentum space in Feynman diagrams obfuscates this simple fact and people start to think that, in the plane wave representation of the momentum space, particles can actually absorb other particles which are located anywhere in the experiment area.

The misunderstanding comes from the fact that if we have independent particles at random locations of the experiment area,  then they can meet at any location and one can absorb the other.


Is newtonian mechanics right about static forces between charges?


In newtonian mechanics, we can get rid of the energy of the electric field by assuming that signals travel infinitely fast. Then the positions of the charges explain everything, and there is no need to assume the existence of an electric field at all, let alone a field which would contain energy.

When we calculate a Feynman diagram where a photon only moves spatial momentum, no energy, then we do assume that the force acts instantaneously. We can drop the time dimension from the model. We are calculating using newtonian mechanics.

What about dynamically changing electromagnetic fields? They do contain energy in on-shell photons, or in virtual photons where the 4-momentum has the energy component non-zero.


An improved rubber plate model of the electromagnetic field


Two years ago, we were able to explain the flow of spatial momentum and energy in a radio transmitter by introducing the model where the electric field of a charge is an elastic "rubber plate".

                             ^
                             |  rubber plate
        ------------------●------------------
                             |  charge
                             v  waved up and down

When an external force waves an electric charge, the rubber plate attached to the charge oscillates and waves in the plate propagate towards infinity.  The waves carry energy but little spatial momentum. The charge exchanges quite a lot of spatial momentum with the rubber plate, as the charge is waved. But the net spatial momentum flow from the charge is zero.

The large gross momentum flow between the plate and the charge explains how the kinetic energy of the charge can be transformed into on-shell photons which carry very little spatial momentum. The charge loses a lot of momentum when it loses its kinetic energy. Where does the extra momentum go? It cannot go to photons.

The answer is that the momentum is temporarily stored in the rubber plate and returned back to the charge when it swings back. The net flow of momentum is zero.

Two years ago we thought that the mass-energy of the static electric field of the charge temporarily absorbs the extra spatial momentum.

But our new hypothesis is that the mass-energy of a static Coulomb field is zero. What to do?

Suppose that the charge is initially static. Then we suddenly start to accelerate it up. Since the speed of light is finite, the plate  has to stretch a little to accommodate the sudden movement of the charge. The rims of the plate are not moving yet but the center is moving up.

We do not need to assume that the plate has any rest mass at all. The stretching will create a force which resists the movement of the charge. The person who is pushing the charge up must give spatial momentum p upward to the plate.

Thus, the finite speed of light can be used to mimic inertial mass, even though the plate is massless.


The birth of electromagnetic waves, or real photons, is understood using Edward M. Purcell's diagram of the electric lines of force bending. See the top of the linked page for a picture how the field lines get wrinkled.

The wrinkle in the rubber plate moves outward. The wrinkle contains both an electric and a magnetic field. That is, the wrinkle is a dynamic electromagnetic field and contains energy.

In Purcell's diagram, the static electric fields can be regarded as containing zero energy. What matters is the energy in the wrinkle.


Scattering of colliding classical electromagnetic waves


A couple of weeks ago we tried to introduce a model of the electromagnetic field where a massless tense string is waved. The waves contain some mass-energy, and will cause inertia in the string. The problem in the model is that then energetic waves should propagate slower than low-energy waves because energetic waves contain a lot of mass-energy and inertia.

What about the rubber plate model? It looks plausible that waving a plate is harder if the plate contains a lot of energy in wrinkles. If there is little energy, then waves propagate precisely at the signal speed of special relativity, that is, c.

But if a wave contains a lot of energy, it should progress slower. Does this sound right?

If two energetic waves C and D collide, then there should probably be scattering of waves because inertia of the wave D disturbs the medium where the wave C propagates. This might be the root cause for the scattering of photons by photons? There would be scattering in classical electromagnetism, after all.

Electromagnetism is nonlinear in QED because energy can escape from the electromagnetic field into the Dirac field. Can we somehow harmonize this fact to the classical scattering of electromagnetic waves?

In Feynman diagrams, photon-photon scattering is modeled as a collision to a virtual pair.

  photon
       _____     _____
                \  /
                 O virtual pair loop
       _____/  \_____
              
   photon

In our classical model, scattering of colliding waves is calculated from the electromagnetic coupling constant (~ tension of a unit electric field line).

In Feynman diagrams, scattering is probably determined by the electron charge / mass ratio and the coupling constant.

If these figures happen to agree, then we have found a formula which determines the electron mass from its charge and the electromagnetic coupling constant.


Yi Liang and Andrzej Czarnecki (2011) write that for low-energy photons, the cross section is proportional to the sixth power of the photon energy. That can hardly happen in our classical model. Conclusion: the classical model cannot explain quantum effects like the electron mass and photon-photon scattering.

Friday, December 18, 2020

The electromagnetic force grows larger at short distances because of van der Waals effects?

https://en.wikipedia.org/wiki/Van_der_Waals_force 

The van der Waals force is believed to exist because transient dipoles in molecules attract each other. It is a polarization force and always attractive, while ordinary polarization is always repulsive.

Suppose that polarization of vacuum is superlinear on the electric field E. We have a good reason to believe that it is superlinear. Very strong electromagnetic forces in particle collisions "tear out" electrons and positrons and they become real particles. Virtual pairs are bound together by the Coulomb force, and that force grows weaker at a greater distance.

                  vacuum
                  polarization
       e- ●     +++   - - -       ● e+
                                      +    -
                                      vacuum
                                      polarization

Suppose that we have an electron and a positron close to each other. The electric field E is double between the charges, and produces some additional vacuum polarization between them because of superlinearity.

The additional dipole of the vacuum polarization strengthens the attraction between the charges.

We need to figure out what is the effect of superlinearity on ordinary, repulsive, vacuum polarization. Does it grow in strength when the charges come closer?


The electron mass is finite because of extreme vacuum polarization close to it?


The classical radius of the electron is 3 * 10^-15 m. The mass-energy of the electric field outside that radius is the electron mass, 511 keV.

Why is the 1 / r^2 electric field massless inside that radius? The reason might be that vacuum polarization almost totally cancels the electric field close to the electron.

There has to be an electric field E inside the classical radius, to maintain vacuum polarization, but the strength of the field E might be of the same order as at, say, the Coulomb force at two classical radii from the electron.

What about the energy which is required to stretch the vacuum dipoles close to the electron? That energy might be very small compared to the infinite energy that is required to create a 1 / r^2 electric field.

A Feynman diagram does not work if an electron passes a nucleus from just one side

This blog post returns to the problem of  "bending" an electron beam under an electric field.

Suppose that we have a static nucleus and a uniform Dirac wave passes symmetrically around it on all sides.

         ------>
      |    |    |    |
      |    |    |    |     ●  Z+ nucleus
      |    |    |    |
      e- electron wave

We can calculate the scattering using the simple Feynman diagram.

    e-  -------------------------
                      |
                      | photon with
                      | momentum p
    Z+ -------------------------

The Fourier decomposition of the Coulomb electric potential 1 / r of Z+ consists of sine wave potentials for each p. The potential undulates in spatial dimensions and is constant in time.

        potential for p component
        ----------------
 e- ----->
        ----------------

        ----------------

In the diagram above, the lines mark the "crests" of the potential of the Fourier component

       1/|p^2|  sin(p • x)

of the Coulomb potential. In the formula, p is the momentum vector of the Fourier component and x is the spatial position vector.

The potential acts as a "diffraction grating" which scatters electrons up and down, giving them the momentum p up or down. The potential crests act as sources for the Dirac equation. These sources have constructive interference for electrons which have "absorbed" the momentum p up or down.

Note that the scattering is both up and down. That is ok because the electron wave arrives symmetric relative to Z+.

        ---->
       |     |     |    ● Z+ nucleus
       |     |     |
       |     |     |
       e- electron wave

If we want to model the scattering for an electron wave which is not symmetric around Z+, then the Feynman diagram method does not work. It predicts scattering both up and down, but the real scattering is just up.

We can calculate the correct scattering pattern with the Schrödinger equation, or even with classical physics.

The Schrödinger calculation can be viewed as a non-perturbative calculation where the passing electron absorbs very many photons from the various Fourier components of the 1 / r potential. The sum of all these photons guides the electron to the correct path up.

The Feynman diagram method is a vastly simplified model where just one photon is exchanged between Z+ and e-. We could call it a minimal quanta method. If the method gives correct answers, it is, of course, much simpler to calculate than the Schrödinger method.

A big question is: why does the Feynman diagram work at all? Why does it work for a symmetric flyby? Is it a coincidence? We have not yet found an answer for that question.


An electron flies past a group of opposite charges


Suppose that we have a bound state of opposite charges. It could be a hydrogen atom, or just some ions attached together somehow. We have the charges static in space. An electron flies by.

The "minimal quanta" method of Feynman diagrams would assume that an electron normally absorbs just one photon, from one of the static charges.

    
                    ● • Z+ e- hydrogen atom
 
      e ----->

The one photon method would give a lot of scattering up and down. But we know that the atom appears electrically neutral to the outside world. There is no scattering.

The experiment has to be calculated using the Schrödinger method. We may imagine that the electron receives many photons from both of the opposite charges. The effects of these photons cancel each other out.

Feynman diagrams do not handle bound states. We have presented one example of that.

We could introduce a tentative rule:

Feynman diagrams are not the correct way to calculate a flyby of charges when the charges do not come "very close". In some (symmetric) cases, Feynman diagrams do work, though. Feynman diagrams do not work for bound states of opposite charges.


                         vacuum
                         polarization
                         loop
       Z+ ~~~~~~~O~~~~~~~ e-
             virtual
             photon

A vacuum polarization loop very much looks like a bound state of opposite charges. Furthermore, the loop is usually "far away" from the electron flying by. There is no reason why a Feynman diagram would model the setup correctly. The divergence of the vacuum polarization integral is a symptom of an incorrect method.


Feynman diagrams do not work if we know something about the relative positions of the particles


The two examples of Feynman diagrams failing have one thing in common: 

The particles in the experiment area are not at random positions. We know something about their relative positions.

In the first example we knew that the electrons pass just on one side of the nucleus.

If there is a bound state, then we know the relative positions of its component particles.

Feynman diagrams assume that free particles collide or fly by at random places in the experiment area. In that case, the single photon formula of Feynman diagrams produces (approximately) correct results.

If there is a collision of particles, say, two photons collide to produce a real pair, then we do know that the initial positions of the electron and positron are close to each other. Apparently, that is not a problem if the produced particles will be at random positions when they interact with other particles the next time.

A vacuum polarization loop contains two particles which are very close to each other for their whole lifetime. We cannot model the process as random collisions of free particles. That is why Feynman diagrams do not work for them.

The same problem as with vacuum polarization, exists for any diagram containing a loop. When the participants of the loop interact again, in most cases they are not at random positions relative to each other.

A correct way to handle bound states in a Feynman diagram might be to invoke the Schrödinger method at certain places. For example, if an electron interacts with a neutral hydrogen atom, we have to assume that the electron always interacts with both of the components of the atom. Or if we have a vacuum polarization loop, then a real electron always interacts with both the electron and the positron in the loop.

The Schrödinger equation calculates the electron wave function under all the potentials which are present. In a Feynman diagram, this is simplified to the one photon method in a special case.

It may be that the divergence in Feynman vacuum polarization is the result of applying the simplified method to a bound state, which is an error.