Stack and concatenate by andrei-papou · Pull Request #735 · rust-ndarray/ndarray · GitHub
Skip to content

Stack and concatenate#735

Closed
andrei-papou wants to merge 13 commits into
rust-ndarray:masterfrom
andrei-papou:stack-and-concatenate
Closed

Stack and concatenate#735
andrei-papou wants to merge 13 commits into
rust-ndarray:masterfrom
andrei-papou:stack-and-concatenate

Conversation

@andrei-papou

@andrei-papou andrei-papou commented Oct 1, 2019

Copy link
Copy Markdown
Contributor

Issue: #667

Renamed stack to concatenate to follow the behaviour of numpy and also added stack function. Examples:

concatenate
use ndarray::{arr2, Axis, concatenate};

let a = arr2(&[[2., 2.], [3., 3.]]);
assert!(
     concatenate(Axis(0), &[a.view(), a.view()])
     == Ok(arr2(&[[2., 2.], [3., 3.], [2., 2.], [3., 3.]]))
);
stack
use ndarray::{arr2, arr3, stack, Axis};

let a = arr2(&[[2., 2.],
               [3., 3.]]);
assert!(
    stack![Axis(0), a, a]
    == arr3(&[[[2., 2.],
               [3., 3.]],
              [[2., 2.],
               [3., 3.]]])
);

Andrew and others added 4 commits April 3, 2019 11:54
Merge main rep master to fork master
Up to date with the main repo master
@bluss

bluss commented Oct 1, 2019

Copy link
Copy Markdown
Member

Comment thread src/stacking.rs Outdated
}
let arrays: Vec<ArrayView<A, D::Larger>> =
arrays.into_iter().map(|a| a.insert_axis(axis)).collect();
concatenate(axis, &arrays)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation strategy wise, can't we take the old stack, put that into an internal function. Give it a parameter that tells it whether to remove an axis or not. It can be a type parameter, since it influences the return type (the dim of the returned array).

That way both stack and concatenate can use this internal function and we don't need the intermediate vector here in concatenate

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! I'll try to implement it this way.

@bluss

bluss commented Oct 2, 2019

Copy link
Copy Markdown
Member

This branch will eventually need to be rebased to skip the merges in the history (see commit view). The blast back to #1, #5, #8 etc in ndarray history is fun, but we don't want to have those merges there.

Also, "this is the right direction" isn't supposed to sound like "nice try". It's supposed to say.. This is the right destination, this is how we want to name those functions. To be pragmatic I think we'd like to have this in 0.13.x, as a non-breaking change in the first round (preparing to put the functions in their right places in the next version).

andrei-papou added 5 commits October 2, 2019 11:02
- got rid of the redundant allocation in `stack_new_axis` implementation
- renamed the functions and macros to make the change backward compatible:
`concatenate` -> `stack`, `stack` -> `stack_new_axis`
Merge main repo master to the fork
@andrei-papou

Copy link
Copy Markdown
Contributor Author

@bluss sorry for the delay, the month is pretty busy. I've refactored the new function to get rid of the redundant allocation. Also I renamed the functions in the backward compatible way. Thanks.

Comment thread src/stacking.rs Outdated
/// [1]: fn.stack.html
///
/// ***Panics*** if the `stack` function would return an error.
/// ***Panics*** if the `concatenate` function would return an error.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

concatenate does not exist yet, we shouldn't refer to it here.

@LukeMathWalker

Copy link
Copy Markdown
Member

If we want to introduce this in 0.13.x, we should probably:

  • mark the existing stack function as deprecated, suggesting to use stack_new_axis;
  • introduce concatenate and concatenate!, which underneath are just calling stack for now;
  • introduce stack_new_axis which will become the "new" stack.

What do you think? @andrei-papou
Always difficult to change what a name means :/

@andrei-papou

Copy link
Copy Markdown
Contributor Author
  • mark the existing stack function as deprecated, suggesting to use stack_new_axis;

@LukeMathWalker shouldn't we suggest to use concatenate instead? concatenate is a new name for the current stack function while stack_new_axis behaves in a different way.

@LukeMathWalker

Copy link
Copy Markdown
Member

Yes, you are right 😅

andrei.papou added 4 commits December 18, 2019 21:54
 - introduced concatenate function
 - deprecated stack function since 0.13.0
 - updated deprecation version for stack function
 - suppressed deprecation warnings
Main repo master to fork master
@andrei-papou

Copy link
Copy Markdown
Contributor Author

@LukeMathWalker ready, but I had to suppress a few deprecation warnings: we still need stack function in concatenate implementation, in lib.rs and in tests.

@andrei-papou

Copy link
Copy Markdown
Contributor Author

@bluss @LukeMathWalker could you please do another round of review? Looks like all the comments have been resolved quite a while ago. Thanks.

Comment thread src/stacking.rs

pub fn stack_new_axis<A, D>(
axis: Axis,
arrays: Vec<ArrayView<A, D>>,

@bluss bluss Apr 18, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stack_new_axis and concatenate should agree on if they take the arrays as a slice or a Vec, they should have the same interface if they can. Avoiding cloning of the array views seems like a minor issue, so it can be decided either way (slice or vec)

Comment thread src/stacking.rs
stack(axis, arrays)
}

pub fn stack_new_axis<A, D>(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing doc comment :)

Comment thread src/stacking.rs
@@ -109,3 +182,71 @@ macro_rules! stack {
$crate::stack($axis, &[ $($crate::ArrayView::from(&$array) ),* ]).unwrap()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The stack macro should also be deprecated

Comment thread src/stacking.rs
res_dim.set_axis(axis, arrays.len());

// we can safely use uninitialized values here because they are Copy
// and we will only ever write to them

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, it follows what we do in stack. We'll revise handling of uninit data, but it's not in scope for this PR. (#796 )

Comment thread src/stacking.rs
let mut res = Array::from_shape_vec(res_dim, v)?;

res.axis_iter_mut(axis)
.zip(arrays.into_iter())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this zip seems to show that we would be fine with just using a slice of ArrayView as input, we don't need to use them by value

Comment thread src/slice.rs
macro_rules! s(
// convert a..b;c into @convert(a..b, c), final item
(@parse $dim:expr, [$($stack:tt)*] $r:expr;$s:expr) => {
(@parse $dim:expr, [$($concatenate:tt)*] $r:expr;$s:expr) => {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This rename of the variable inside the macro is an incidental mistake, and should not be in the PR

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I removed this in my rebase-fix of the branch. No other changes in that branch. :)

@bluss

bluss commented Apr 21, 2020

Copy link
Copy Markdown
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants