![]() %timeit sum_matrix_sparse_np(x_row_np, x_col_np, x_val_np) Sum_matrix_sparse_np(x_row_np, x_col_np, x_val_np) # Here you could use i and j in your real code as well of course X_val_np = sum_matrix_sparse_np(rows, cols, vals): Here is an example using 3 arrays, but maybe one could also use a single “structured array”, if that feels more natural, and it could be even faster. Putting the same exact rows/cols/values into tuples in a list just makes the data structure more complex than required. However, I think you don’t really need a list here, it is perfectly fine to pass the numpy arrays for cols rows and values into a function and iterate over them directly. ![]() This is probably made even worse by how this may affect memory caching in the CPU. A list will always require jumping through possibly non sequential memory and will have one or more levels of indirection, meaning the computer first needs to find the address where a value is stored before it can find the value. And that is not due to numba, but simply due to how list and arrays usually work on the hardware level.Īrray memory is normally sequential which means the computer can jump directly to a known memory address and read the value. Hi think you will never see a conventional list perform as fast as an array. Is there a better / faster way of using read-only Python lists (or even a list of tuples) with Numba jit than I have done above? Will eventually run just as fast with Python lists as Numpy arrays, so I can use a Python list or even a Python list of tuples directly with my Numba jit functions? Or will it most likely always be much faster to pass Numpy arrays to Numba jit functions? It seems that in the current version of Numba (0.54.1), the fastest method by far, is to pass Numpy arrays instead of Python lists to Numba jit functions. # This is very fast when using separate Numpy arrays as the arguments. %timeit sum_matrix_sparse2(x_col_list=x_col_np, x_row_list=x_row_np, x_val_list=x_val_np) # This runs without a warning but is much slower than 'sum_matrix_dense()'. %timeit sum_matrix_sparse2(x_col_list=(x_col_list), x_row_list=(x_row_list), x_val_list=(x_val_list)) # 5.69 ms ± 2.14 ms per loop (mean ± std. # 'x_col_list' of function 'sum_matrix_sparse2'. %timeit sum_matrix_sparse2(x_col_list=x_col_list, x_row_list=x_row_list, x_val_list=x_val_list) of 7 runs, 1000 loops sum_matrix_sparse2(x_col_list, x_row_list, x_val_list): # This runs without warnings but is much slower than 'sum_matrix_dense()'. %timeit sum_matrix_sparse1(x_list_tup=(x_list_tup)) # 24.7 ms ± 1.58 ms per loop (mean ± std. # 'x_list_tup' of function 'sum_matrix_sparse1'. %timeit sum_matrix_sparse1(x_list_tup=x_list_tup) ![]() # but in my real use-case I need those as well. # This is a bit contrived because the indices i and j are ignored, of 7 runs, 100000 loops sum_matrix_sparse1(x_list_tup): ![]() X_list_tup = list(zip(x_col_list, x_row_list, sum_matrix_dense(x): # Convert to a list of tuples, which is the format I prefer in my use-case. # Numpy arrays for the column and row indices, and the matrix values. # Convert the sparse matrix to a dense matrix. X_sparse = (n, n, density=0.1, format='coo') # Generate a sparse matrix with random elements. This does not seem to be supported very well in Numba as it runs very slowly.īelow is a somewhat contrived example which just sums all the elements of a sparse matrix and could therefore ignore the coordinates for the sparse matrix, but my actual problem needs to use the coordinates of the sparse matrix as well. In my actual use-case I have made a function that operates on a sparse matrix, whose elements are defined in a list of tuples like this: where the iX and jX are integer-values for the coordinates into the sparse matrix and vX are the float values for the matrix-elements at those coordinates. The slowest is to use the new and experimental. The fastest method by far is passing a Numpy array. # scheduled for deprecation: type 'reflected list' found for argument # NumbaPendingDeprecationWarning: Encountered the use of a type that is Let me first show the problem with a simple example, where we just sum all the elements of a list: # Number of elements in the list. My question relates to the best practices of passing read-only Python lists to Numba jit functions. Thanks for making Numba, I am a big fan! I’ve got the Numba fan-poster, t-shirt, cap, underwear and everything, even my bed-sheets say Numba! (OK, that’s not really true, but it could be!)
0 Comments
Leave a Reply. |