-
Notifications
You must be signed in to change notification settings - Fork 3.1k
Add support for Views to IsIterableLike #6674
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
} | ||
|
||
testIterableOps | ||
testIterable |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I removed the testIterable
part which was just duplicating what we had in testIterableOps
. And I added more test cases to the testIterableOps
(to test with Views and Maps).
bc6cfc0
to
ff003be
Compare
Note: after we merge this PR we should update that section of the FAQ to say that it’s not anymore needed to use |
ff003be
to
8b0feb5
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, but I'd like @szeiger to review it, too.
} | ||
|
||
object IsSeqLike { | ||
import scala.language.higherKinds | ||
|
||
implicit val stringRepr: IsSeqLike[String] { type A = Char } = | ||
implicit def seqOpsIsIterableLike[CC0[X] <: SeqOps[X, Iterable, CC0[X]], A0]: IsSeqLike[CC0[A0]] { type A = A0; type C = CC0[A0] } = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should this be named seqOpsIsSeqLike
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes.
} | ||
|
||
implicit def rangIsSeqLike[C0 <: Range]: IsSeqLike[C0] { type A = Int; type C = immutable.IndexedSeq[Int] } = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this needed / why isn't seqOpsIsSeqLike
enough?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because seqOpsIsSeqLike
provides an instance that matches a type constructor CC0[X]
with one type parameter. Range
takes no type parameter so it can never be unified with CC0[X]
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks. Don't we have the same issue with BitSets?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed, good catch! I’ve updated the PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, but then also LongMap? Others?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed. :-/
But unfortunately a generic definition like the following doesn’t work:
implicit def iterableIsIterableLike[Repr <: Iterable[A0], A0]: IsIterableLike[Repr] { type A = A0; type C = Iterable[A0 } = …
Beause scalac always infers Nothing
for A0
:
type arguments [LongMap[String],Nothing] do not conform to method iterableIsIterableLike's type parameter bounds [Repr <: Iterable[A0],A0]
Maybe there is a trick to get A0
correctly inferred?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@julienrf I don't know if this is still useful to anyone, but this?
implicit def iterableIsIterableLike[Repr <: Iterable[A0], A0]: IsIterableLike[Repr with Iterable[A0]] { type A = A0; type C = Iterable[A0] } = ???
45a9556
to
96d0ef9
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not convinced by these additions. Maybe @milessabin as the original author of IsTraversableLike
would like to weigh in on this PR.
} | ||
|
||
// 2. Sorted Map collections | ||
implicit def sortedMapIsImmutableMapLike[CC0[X, +Y] <: immutable.Map[X, Y] with immutable.SortedMapOps[X, Y, CC0, CC0[X, Y]], K0, V0] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this necessary? SortedMap
extends Map
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because CC0[X, Y] <: SortedMap[X, Y]
gives us CC0[X, Y] <: MapOps[X, Y, Map, CC0[X, Y]]
, whereas the mapIsImmutableMapLike
definition requires CC0[X, Y] <: MapOps[X, Y, CC0, CC0[X, Y]]
(note the third type parameter applied to MapOps
).
type A | ||
/** A conversion from the representation type `Repr` to `IterableOps[A, Iterable, Repr]`. */ | ||
val conversion: Repr => IterableOps[A, Iterable, Repr] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IsTraversableLike
in 2.12 calls it conversion
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I can call it back conversion
. I think I initially made it a method following up that comment.
* | ||
* @tparam Repr Collection type (e.g. `Map[Int, String]`) | ||
*/ | ||
trait IsMapLike[Repr] extends IsIterableLike[Repr] { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not clear to me what functionality this adds. IsIterableLike
and IsSeqLike
can abstract over String
and Array
in addition to collections. There is no such case for maps.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is MapView
that doesn’t extend Map
.
} | ||
|
||
// 2. MapView | ||
implicit def mapViewIsMapLike[CC[X, Y] <: MapView[X, Y], K0, V0]: IsMapLike[CC[K0, V0]] { type K = K0; type V = V0; type C = View[(K, V)] } = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is MapView
treated specially? MapView
extends MapOps
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the same reason as with SortedMap
above.
Marked it as WIP for now; @julienrf can you investigate the options for writing extension methods and bring them in as test cases? |
78c6438
to
1ad3a53
Compare
I’ve come up with a quite complicated but powerful design. This API targets advanced users so maybe the complexity is worth it… First, let me restate the goal of this work: let users define extension methods on collections in a generic way (so that the operation can be applied to a We already have something called The solution explored in this PR is to generalize trait IsIterableLike[Repr] {
type A
type C
def apply(coll: Repr): IterableOps[A, Iterable, C]
} If you look at the tests you will see that it makes it possible to implement quite complicated scenarios, like the trait SeqOps[A, CC[_], C] {
def splitWith(p: A => Boolean): CC[CC[A]]
} This operation returns an |
1ad3a53
to
e76239b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I generally agree about having this in its complexity. The documentation of IsIterableLike should also mention / link to less generic ways of extending collections (We have / will have that in on the doc site, right?).
It's not clear to me why some parts of the collections hierarchy are mirrored here (immutable.Map
) but not others.
For the SortedMap speical case, (why) don't we have the same issue with SortedSet?
There are still various missing instances: LongMap
, IntMap
, AnyRefMap
. Also:
scala> class ExtensionMethods[Repr, I <: collection.generic.IsIterableLike[Repr]](coll: Repr, it: I) { def foo = 1 }
scala> implicit def withExtensions[Repr](coll: Repr)(implicit it: collection.generic.IsIterableLike[Repr]):
scala> List().foo
res0: Int = 1
scala> Nil.foo
^
error: value foo is not a member of object Nil
@@ -87,55 +85,70 @@ import scala.reflect.ClassTag | |||
* instance of `IsIterableLike` specific to the new type. | |||
* | |||
* Below is an example of an implementation of the `IsIterableLike` trait | |||
* where the `Repr` type is `String`. | |||
* where the `Repr` type is `Range`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the current form, I think this is not the best example because the actual implicit for Range
implements IsSeqLike
.
I think it would make sense to first talk about IsSeqLike
and IsMapLike
here, and say why they exist. Then it's good to give rangeIsSeqLike
as an example (and say that this is how it's actually implemented).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ping ☝️
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The goal of this documentation is not to explain how things actually work, but how users can implement IsIterableLike
instances. So, I think it’s fine to use Range
as an example.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, but then add a comment saying that this is an example, the actual implementation for ranges exists and doesn't need to be written by the user, and it uses IsSeqLike
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
* has keys of type `K`, values of type `V` and has a conversion to `MapOps[K, V, _, _]`. | ||
* | ||
* This type enables simple enrichment of `Map`s with extension methods. | ||
* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add @see [[scala.collection.generic.IsIterableLike]]
// `Range` can not be unified with the `CC0` parameter of the | ||
// `seqOpsIsSeqLike` definition because it does not take a type parameter. | ||
// Hence the need for a separate case: | ||
implicit def rangIsSeqLike[C0 <: Range]: IsSeqLike[C0] { type A = Int; type C = immutable.IndexedSeq[Int] } = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
typo rang**e**IsSeqLike
import scala.language.higherKinds | ||
|
||
/** | ||
* Type class witnessing that a collection type `C` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
C
-> Repr
?
|
||
type A = (K, V) | ||
|
||
/** A conversion from the type `C` to `MapOps[K, V, _, _]` */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
C
-> Repr
?
def apply(coll: CC0[A0]): SeqOps[A0, View, View[A0]] = coll | ||
} | ||
|
||
implicit def stringIsSeqLike: IsSeqLike[String] { type A = Char; type C = String } = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
could be a val
} | ||
|
||
object IsSeqLike { | ||
import scala.language.higherKinds | ||
|
||
implicit val stringRepr: IsSeqLike[String] { type A = Char } = | ||
implicit def seqOpsIsSeqLike[CC0[X] <: SeqOps[X, Iterable, CC0[X]], A0]: IsSeqLike[CC0[A0]] { type A = A0; type C = CC0[A0] } = | ||
new IsSeqLike[CC0[A0]] { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you use a single instance and a cast for these methods?
def apply(c: CC0[K0, V0]): immutable.MapOps[K0, V0, MapCC, C] = c | ||
} | ||
|
||
// 2. Sorted Map collections |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add a comment why this is needed
val splitted9 = "foo".splitWith(_.isLower) | ||
val splitted9T: IndexedSeq[String] = splitted9 | ||
val splitted10 = "foo".view.splitWith(_.isLower) | ||
val splitted10T: View[View[Char]] = splitted10 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 for testing type inference
There is no good reason for that, actually. I implemented these classes when working on extension methods, in the I’m not sure we should mirror each level of the hierarchy, though. For sure,
We don’t need it because both
Yes, it’s unfortunate but if we try to create a more general case like the following: def foo[C <: IterableOps[A, Iterable, C], A]: IsIterableLike[C] = … (which should theoretically work for It doesn’t work due to type inference issues. The type inferencer is unable to correctly instantiate the type parameter That’s why we have to add special cases for |
cac00af
to
8b0db52
Compare
Following up our discussion I’ve removed So, in this PR, we have:
I think this is good to go so I’m removing the “wip“ label. |
I'd go with |
OK, so let's remove IsMapLike, make sure the documentation shows how to write extension methods on maps, and rename to IsIterable also sounds good to me. @julienrf do you have time to do that? |
I can probably do it before the end of next week. |
} | ||
b.result() | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I’ve tried to write a decorator that uses MapOps
instead of IsMapLike
but it doesn’t work:
implicit class MapDecorator[M <: MapOps[K, V, CC, C], K, V, CC[X, Y] <: IterableOps[_, AnyConstr, _], C](coll: M) {
def leftOuterJoin[W, That](other: Map[K, W])(implicit bf: BuildFrom[M, (K, (V, Option[W])), That]): That = {
val b = bf.newBuilder(coll)
for ((k, v) <- coll) {
b += k -> (v, other.get(k))
}
b.result()
}
def rightOuterJoin[W, That](other: Map[K, W])(implicit bf: BuildFrom[M, (K, (Option[V], W)), That]): That = {
val b = bf.newBuilder(coll)
for ((k, w) <- other) {
b += k -> (coll.get(k), w)
}
b.result()
}
}
The implicit conversion is never triggered. If I try to explicitly call the conversion, I get type errors like so:
[error] inferred type arguments [scala.collection.immutable.Map[Int,String],Nothing,Nothing,Nothing,Nothing] do not conform to method MapDecorator's type parameter bounds [M <: scala.collection.MapOps[K,V,CC,C],K,V,CC[X, Y] <: scala.collection.IterableOps[_, collection.AnyConstr, _],C]
[error] val mapLJoin = MapDecorator(map).leftOuterJoin(Map(1 -> "bar"))
[error] ^
I think users will need to define an intermediate type for the type unification to work. This is exactly what IsMapLike
does, so why not just keep it instead of telling users to copy it everywhere?
fbe9065
to
3403e0c
Compare
You don't need implicit class MapDecorator[K, V, CC[_, _] <: IterableOps[_, Any, _], C](coll: MapOps[K, V, CC, C] with C) { |
@szeiger is your objection a blocker for M5, or can we merge this for M5 and then fix it for RC1? |
3403e0c
to
b68645c
Compare
I rebased this on current 2.13.x. Note that the |
@lrytz will look into it, thanks |
I pushed a commit that reverts 6644d68, but just for testing. And I pushed a commit that removes For reference, the reason why |
I think maybe we should revert 6644d68 for now, since that problem is quite a bit less important than this pull request, and it seems like a lot of tests introduced in this ticket break |
016479d
to
3a35468
Compare
type C = View[(K0, V0)] | ||
def apply(coll: CC0[K0, V0]): IterableOps[A, Iterable, View[(K0, V0)]] = coll | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We still need explicit IsIterable
instances for maps, I assume due to MapOps
having a different number of type parameters. I added the two instances here that should make existing tests pass. This is however incomplete, we also need instances for AnyRefMap, IntMap, LongMap.
So in the end we don't gain much by removing IsMap
in terms of lines of code. In this case, I have a preference to keep it.
@julienrf @SethTisue @szeiger what do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it’s simpler to keep IsMap
. Otherwise, we can check carefully that all the previous IsMap
instances now have a corresponding IsIterable
instance.
I think this solution works by chance. For instance, for |
OK, I think we should merge #7088, then rebase this PR and get it in with your two commits (include |
#7088 is merged |
The conversion from the `Repr` type now returns an `IterableOps[A, CC, C]`, whereas it previously had to return an `IterableOps[A, Iterable, Repr]`. This change makes it possible to create instances of `IsIterableLike` for View collections. Also, the `conversion` member has been refactored into an `apply` method. In addition to `IsIterableLike` and `IsSeqLike`, the type `IsMapLike` has been introduced. Last, the `IsXxxLike` types now form a hierarchy: `IsMapLike` specializes `IsSeqLike`, which itself specializes `IsIterableLike`. Fixes scala/collection-strawman#535
3a35468
to
57a8ba4
Compare
// Makes `IsMap` instances visible in `IsIterable` companion | ||
implicit def isMapLikeIsIterable[Repr](implicit | ||
isMapLike: IsMap[Repr] | ||
): IsIterable[Repr] { type A = isMapLike.A; type C = isMapLike.C } = isMapLike |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rename to isMapIsIterable
? 🤔
implicit def isMapIsIterable[Repr](implicit
isMap: IsMap[Repr]
): IsIterable[Repr] { type A = isMap.A; type C = isMap.C } = isMap
// Makes `IsSeq` instances visible in `IsIterable` companion | ||
implicit def isSeqLikeIsIterable[Repr](implicit | ||
isSeqLike: IsSeq[Repr] | ||
): IsIterable[Repr] { type A = isSeqLike.A; type C = isSeqLike.C } = isSeqLike |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same as isMapIsIterable
specifically scala/scala#6674
The conversion from the
Repr
type now returns anIterableOps[A, CC, C]
,whereas it previously had to return an
IterableOps[A, Iterable, Repr]
.This change makes it possible to create instances of
IsIterableLike
for View collections.
Also, the
conversion
member has been refactored into anapply
method.In addition to
IsIterableLike
andIsSeqLike
, the typesIsMapLike
and
IsImmutableMapLike
have been introduced.Last, the
IsXxxLike
types now form a hierarchy:IsImmutableMapLike
specializesIsMapLike
, which itself specializesIsIterableLike
.