Adds prod and hstack/vstack and fixes an issue with the newest version of NumPy

[PTNobel]

Description

Nothing huge, just canon, Jacobian, and Hessian_vec support for cp.prod. Also added Jacobian to hstack/vstack (and therefore bmat).

[github-actions]

Benchmarks that have stayed the same:

   before           after         ratio
 [496bbf35]       [12af971c]
      2.77±0s          2.82±0s     1.02  quantum_hilbert_matrix.QuantumHilbertMatrix.time_compile_problem
      1.83±0s          1.85±0s     1.01  simple_QP_benchmarks.UnconstrainedQP.time_compile_problem
      21.9±0s          22.1±0s     1.01  sdp_segfault_1132_benchmark.SDPSegfault1132Benchmark.time_compile_problem
      5.05±0s          5.08±0s     1.01  svm_l1_regularization.SVMWithL1Regularization.time_compile_problem
      660±0ms          664±0ms     1.01  matrix_stuffing.ConeMatrixStuffingBench.time_compile_problem
      288±0ms          289±0ms     1.00  slow_pruning_1668_benchmark.SlowPruningBenchmark.time_compile_problem
      1.05±0s          1.06±0s     1.00  finance.FactorCovarianceModel.time_compile_problem
      1.36±0s          1.36±0s     1.00  matrix_stuffing.ParamConeMatrixStuffing.time_compile_problem
      4.90±0s          4.91±0s     1.00  optimal_advertising.OptimalAdvertising.time_compile_problem
      164±0ms          164±0ms     1.00  high_dim_convex_plasticity.ConvexPlasticity.time_compile_problem
      4.47±0s          4.48±0s     1.00  huber_regression.HuberRegression.time_compile_problem
     14.3±0ms         14.3±0ms     1.00  simple_LP_benchmarks.SimpleFullyParametrizedLPBenchmark.time_compile_problem
      11.1±0s          11.1±0s     1.00  simple_LP_benchmarks.SimpleLPBenchmark.time_compile_problem
      280±0ms          279±0ms     1.00  matrix_stuffing.ParamSmallMatrixStuffing.time_compile_problem
     38.6±0ms         38.4±0ms     1.00  matrix_stuffing.SmallMatrixStuffing.time_compile_problem
      255±0ms          253±0ms     0.99  simple_QP_benchmarks.SimpleQPBenchmark.time_compile_problem
      12.7±0s          12.6±0s     0.99  finance.CVaRBenchmark.time_compile_problem
      520±0ms          516±0ms     0.99  semidefinite_programming.SemidefiniteProgramming.time_compile_problem
      1.11±0s          1.10±0s     0.99  gini_portfolio.Cajas.time_compile_problem
      918±0ms          909±0ms     0.99  simple_LP_benchmarks.SimpleScalarParametrizedLPBenchmark.time_compile_problem
      820±0ms          810±0ms     0.99  simple_QP_benchmarks.LeastSquares.time_compile_problem
      1.68±0s          1.66±0s     0.98  tv_inpainting.TvInpainting.time_compile_problem
      336±0ms          331±0ms     0.98  gini_portfolio.Yitzhaki.time_compile_problem
     14.5±0ms         14.2±0ms     0.98  simple_QP_benchmarks.ParametrizedQPBenchmark.time_compile_problem
      235±0ms          227±0ms     0.97  gini_portfolio.Murray.time_compile_problem

[PTNobel]

The new NumPy version breaks our tests.

Because of this expired deprecation: https://numpy.org/doc/stable/release/2.4.0-notes.html#raise-typeerror-on-attempt-to-convert-array-with-ndim-0-to-scalar

I can't tell if it is in IPOPT or us.

[PTNobel]

The bug was on our end, we were returning a (1,) from oracles.objective in some cases. I fixed it by accessing the .value of the objective directly, instead of getting that of its first arg.

[Transurgeon]

The bug was on our end, we were returning a (1,) from oracles.objective in some cases. I fixed it by accessing the .value of the objective directly, instead of getting that of its first arg.

very nice fix, thanks for doing that!

[Transurgeon]

I think this is looking good.
I only looked at the implementation of hstack and it makes a lot of sense.
I also double-checked all tests and they also seem to be correct.
The implementation of vstack is a bit confusing, but that's normal since we are considering fortran order.

[dance858]

Awesome, thank you for doing this! Two of the thoughts that popped up as I skimmed through hstack and vstack.

1. Why do we update flat_offset in, say, jacobian of hstack in such a complicated way? Can't we just always add arg_size? Do we really need to special case the 1d case like we currently do? Please check all functions if it is really necessary.
2. Can there really be duplicates in the jacobian of hstack? Is the loop that sums the duplicates really necessary?

I think we should be careful so we are not sloppy and put unnecessary AI generated code in the repo.

[PTNobel]

Why do we update flat_offset in, say, jacobian of hstack in such a complicated way? Can't we just always add arg_size?

Actually, we don't even need to compute arg_size in such a complex way, arg.size works

Do we really need to special case the 1d case like we currently do? Please check all functions if it is really necessary.

Fair point!

Can there really be duplicates in the jacobian of hstack? Is the loop that sums the duplicates really necessary?

Ah there cannot be duplicates in either Jacobian. There can be in the Hessians (e.g. hstack([cp.log(x), cp.exp(x)]) ends up with duplicates).

[dance858]

can you add so if args[0] is already a variable we just return, see for example exp_canon

[dance858]

Nice changes! I think vstack and hstack look neat. One possible refactor to be consistent with the rest of the code: in, for example, the jacobian of add_expr we have a list with keys_that_require_summing. Can we add that to the hessians of vstack and hstack? After this I'm happy with vstack and hstack. Good job!

I'm also fine with the product code, even though I haven't gone through it carefully to understand if there's an easier and simpler way.

Let's merge it once you have fixed the small comments above. Thanks!

[PTNobel]

The product code complexity comes from trying to make it impossible for a division by zero